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Chapter  1 


INTRODUCTION 


Overview 

Current  contracting  directives  establish  an  objec-  • 
tive  of  increased  competition  between  similar  or  differing 
systems  design  concepts  throughout  the  entire  acquisition 
process  of  a  major  system  (9**0  •  Along  with  these  direc¬ 
tives,  policies  have  been  issued  which  are  designed  to  in¬ 
crease  off-the-shelf  buys  and/or  modifications  of  off-the- 
shelf  items  in  order  to  decrease  the  cost  of  research  and 
development  and  increase  reliability  and  accessibility  (11) 
These  directives  and  policies  result  in  requests  by  the 
government  for  technical  information  at  various  stages  in 
the  acquisition  process.  This  information  is  frequently 
the  subject  of  disagreement  between  the  contractor  and  the 
government  due  to  the  contractor's  perception  that  the  data 
requested  are  sensitive  and,  in  some  cases,  not  necessary 
for  the  intended  use  of  the  end  item.  The  government  may 
be  experiencing  increased  reluctance  by  contractors  to 
grant  the  Air  Force  substantial  rights  in  data  that  could 
threaten  the  contractor's  perceived  competitive  edge  in  com 
mercial  and  defense  programs.  This  situation  may  also  be 
causing  increased  expenditures  of  time  and  effort  by  the 


principal  contracting  officer  (PCO)  on  negotiation  and  in¬ 
creased  cost  of  end  items  to  the  government. 


Statement  of  the  Problem 

What  are  the  relationships,  if  any,  between  govern¬ 
ment  ownership  of  source  selection  sensitive  data  and  the 
following  variables: 

(1)  the  level  of  competition, 

(2)  the  time  and  cost  of  negotiation  effort,  and 

(3)  the  level  of  incentives? 

If  relationships  exist  which  have  the  effects  of  decreased 
competition  and/or  increased  time  and  cost  of  contract 
award,  what  incentives  and  protection  may  be  offered  to  the 
contractor  to  alleviate  the  problem? 

Background 

The  purpose  of  the  defense  acquisition  process  is 
to  develop  and  supply  the  weapons,  equipment,  and  services 
required  to  meet  the  United  States  national  defense  objec¬ 
tives.  The  major  products  of  the  defense  industry  are 
described  as  "weapon  systems."  This  term  includes  not  only 
technically  complex  items  such  as  aircraft,  missiles,  and 
tanks  but  also  the  techniques,  hardware,  and  subsystems 
needed  to  operate  and  support  the  major  items.  "Systems 

*For  the  purposes  of  this  thesis,  the  term  "con¬ 
tracting  officer"  refers  to  the  person  in  the  contracting 
office  having  responsibility  for  the  effort.  It  could  be 
either  the  principal  contracting  officer  (PCO)  or  the  buyer. 


program  management"  is  the  name  of  the  process  by  which 
weapon  systems  are  acquired.  This  term  includes  all  man¬ 
agement  activity  necessary  to  enable  governmental  agencies 
and  their  contractors  to  accomplish  a  program's  objectives 
(511*0. 

The  acquisition  process  includes  two  stages.  The 
first  stage  includes  the  responsibilities  of  planning,  re¬ 
search,  development,  testing,  and  evaluation.  The  second 
stage  is  production.  Within  the  Air  Force,  the  first  stage 
is  carried  out  by  the  Air  Force  Systems  Command  (AFSC)  and 
the  second  stage  is  the  responsibility  of  the  Air  Force 
Logistics  Command  (AFLC).  AFSC  acquisition  functions  in¬ 
clude  purchasing  of  design  studies,  research  and  develop¬ 
ment,  prototypes,  engineering  development,  and  initial  pro¬ 
duction.  The  AFLC  acquisition  f unctions  consist  of  pur¬ 
chasing  necessary  support  to  operate  and  maintain  the  sys¬ 
tem  throughout  its  life. 

The  acquisition  process  in  both  commands  includes 
purchase  request  review,  solicitation,  evaluation,  and 
award.  This  process  is  carried  out  in  the  Department  of 
Defense  (DOD)  by  either  formal  advertising  or  negotiation. 
The  Defense  Acquisition  Regulation  (DAR)  recommends  formal 
advertising  as  the  preferred  method  of  contracting.  How¬ 
ever,  only  12  percent  of  the  $18  billion  awarded  for  Fiscal 
Year  (FY)  78  by  the  Air  Force  was  awarded  through  formal 
advertising  (2t 32-37).  Negotiation  resulted  in  approxi¬ 
mately  $16  billion  spent  by  the  Air  Force  in  FY  78. 
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Under  some  circumstances,  it  is  not  practical  to 
conduct  government  procurements  through  formal  advertising. 
In  the  area  of  research  and  development,  contracts  are 
normally  awarded  through  negotiation  allowing  the  government 
to  evaluate  the  technical  and  management  capabilities  of  the 
contractors,  as  well  as  the  costs.  Negotiations  also  allow 
the  government  more  flexibility.  Nonetheless,  a  procure¬ 
ment  may  only  be  conducted  through  negotiation  if  it  falls 
into  one  of  the  following  17  categories  and  written  per¬ 
mission  has  been  received  from  the  appropriate  official  — 
commonly  the  Secretary  of  a  military  departments 

1.  A  national  emergency; 

2.  Public  exigency; 

3.  Purchases  not  in  excess  of  $10,000; 

4.  Personal  or  professional  services; 

5.  Services  of  educational  institutions; 

6.  Purchases  outside  the  United  States; 

•  ?.  Medicines  or  medical  supplies; 

8.  Supplies  purchased  for  authorized  resale; 

9*  Perishable  or  nonperishable  subsistence 
supplies; 

10.  Supplies  or  services  for  which  it  is  im¬ 
practical  to  secure  competition  by  formal 
advertising; 

11.  Experimental,  developmental,  or  research 
work; 

12.  Classified  purchases; 

13*  Technical  equipment  requiring  standard¬ 
ization  and  intercnange ability  of  parts; 

l4.  Technical  or  specialized  supplies  re¬ 
quiring  substantial  initial  investment  or 
extended  period  of  preparation  for  manu¬ 
facture; 

15*  Negotiation  after  advertising; 

16.  Purchases  in  the  interest  of  national  de¬ 
fense  or  industrial  mobilization; 

17*  Procurement  otherwise  authorized  by  law, 
e.g.,  architectural  or  engineering 
services  for  preparing  specifications  for 
public  works,  utilities,  naval  vessels, 
or  aircraft  construction  [5»253]* 
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Under  negotiation,  awards  can  be  either  sole  source 
2 

or  competitive  .  Office  of  Management  and  Budget  (OMB) 
Circular  No.  A-109,  issued  5  April  1976,  directs  the  PCO  to 
emphasize  "competitive  exploration  of  alternatives,  as  re¬ 
latively  inexpensive  insurance  against  premature  or  pre¬ 
ordained  choice  of  a  system  that  may  prove  to  be  either  more 
costly  or  less  effective  [9»3]."  Historically,  competition 
in  the  acquisition  of  defense  hardware  has  been  very  diffi¬ 
cult  to  arrange  and  to  enforce.  Competition  relies  on  the 
abilities  of  the  contractors  to  design  and  produce  the  items 
that  best  meet  the  requirements.  The  difficulties  in  se¬ 
curing  adequate  competition  have  arisen  in  government  con¬ 
tracting  primarily  because  each  weapon  system  must  be  de¬ 
signed  for  a  particular  purpose  thereby  incorporating  spe¬ 
cial  features  of  creative  engineering,  technical  innovations, 
and  advanced  manufacturing  techniques.  If  the  government 
^affT'  assume  effective  ownership  of  these  special  features,  it 
may  disseminate  the  special  designs  to  industry  and  procure 
the  hardware  from  other  manufacturers.  However,  if  the  con¬ 
tractor  modifies  his  own  design  which  was  developed  at  his 
own  expense,  he  may  retain  the  right  to  a  significant  part 
of  the  hardware  data  thereby  locking  himself  into  a  sole 

source  position  as  long  as  that  design  is  effective  (8tl). 

- - - 

The  term  "competitive”  may  or  may  not  include  price 
competition.  Competition  may  be  based  on  design  excellence, 
responsiveness  to  Government  requests,  or  overall  quality  of 
contractor  proposals,  rather  than  price  [5*2593* 
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The  level  of  competition,  defined  for  the  purposes  of  this 
thesis  as  the  number  of  offers  received  in  response  to  the 
request  for  proposals  (RFP),  may  be  affected  significantly 
by  the  contractor's  ability  to  retain  the  data  rights  to 
his  design  either  through  the  use  of  patents  or  proprietary 
rights  in  data. 

The  distinction  between  proprietary  data  and  patents 
is  significant.  A  "patent"  is  an  explicit  license  to  place 
a  restriction  upon  the  use  of  the  knowledge  or  techniques 
after  full  disclosure.  The  patent  is  protected  by  a  limited 
scope  and  an  enforceable  claim.  Proprietary  rights  in  data 
based  upon  "trade  secrets"  depend  on  nondisclosure.  If  the 
government  uses  contractor  data  to  increase  competition  for 
specially  designed  equipment,  it  is  necessary  to  disclose 
the  details  of  design  and  production  to  potential  competi¬ 
tors.  If  the  disclosure  includes  data  for  privately-devel¬ 
oped  concepts  utilized  in  the  contractor's  commercial 
business,  the  problem  is  compounded.  It  is  not  difficult 
to  comprehend  the  radically  differing  views  on  proprietary 
data  by  contractors  and  the  government  (8*1). 

"Technical  data"  is  defined  as  the  means  for  commu¬ 
nication  of  concepts,  plans,  descriptions,  requirements ,  and 
instructions  relating  to  technical  projects.  The  govern¬ 
ment's  policy  is  to  acquire  only  data  and  rights  essential 
to  meeting  the  government's  needs.  The  data  may  be  acquired 
with  limited  or  unlimited  rights.  Data  acquired  with 
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unlimited  rights  can  be  used,  duplicated,  or  disclosed  by 
the  government  for  any  purpose.  Data  acquired  with  limited 
rights  can  be  used  freely  within  the  government  but  cannot 
be  disclosed  to  a  third  party.  The  determining  factor  in 
the  decision  between  limited  and  unlimited  rights  is  whether 
the  data  are  developed  at  private  or  government  expense. 

Data  developed  at  private  expense  are  normally  acquired  with 
limited  rights,  and  data  developed  at  government  expense  are 
normally  acquired  with  unlimited  rights  (6). 

In  keeping  with  government  data  policy,  technical 
data  in  the  following  categories  are  normally  acquired  with 
unlimited  rights t 

(i)  technical  data  and  computer  software  re¬ 
sulting  directly  from  performance  of  experimental, 
developmental  or  research  work  which  was  specified 
as  an  element  of  performance  in  this  or  any  other 
Government  contract  or  subcontract; 

(ii)  computer  software  required  to  be  originated 
or  developed  under  a  Government  contract,  or  gener¬ 
ated  as  a  necessary  part  of  performing  a  contract; 

(iii)  computer  data  bases,  prepared  under  a 
Government  contract,  consisting  of  information 
supplied  by  the  Government,  information  in  which 
the  Government  has  unlimited  rights,  or  informa¬ 
tion  which  is  in  the  public  domain; 

(iv)  technical  data  necessary  to  enable  manu¬ 
facture  of  end-items,  components  and  modifications, 
or  to  enable  the  performance  of  processes,  when  the 
end-item,  components,  modifications  or  processes 
have  been,  or  are  being,  developed  under  this  or 
any  other  Government  contract  or  subcontract  in  which 
experimental,  developmental  or  research  work  is, 
or  was  specified  as  an  element  of  contract  perfor¬ 
mance,  except  technical  data  pertaining  to  items, 
components,  processes,  or  computer  software  developed 
at  private  expense.  .  .  ; 
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(v)  technical  data  or  computer  software  pre¬ 
pared  or  required  to  be  delivered  under  this  or  any 
other  Government  contract  or  subcontract  or  con¬ 
stituting  corrections  or  changes  to  Government- 
furnished  data  or  computer  software; 

(vi)  .technical  data  pertaining  to  end-items; 
components  or  processes ,  prepared  or  required  to  be 
delivered  under  this  or  any  other  Government  con¬ 
tract  or  subcontract,  for  the  purpose  of  identifying 
source,  size,  configuration,  mating  and  attachment 
characteristics,  functional  characteristics  and 
performance  requirements  ("form,  fit  and  function" 
data,  e.g.,  specification  control  drawings,  catalog 
sheets,  envelope  drawings,  etc.); 

(vii)  manuals  or  instructional  materials  prepared 
or  required  to  be  delivered  under  this  contract  or 
any  subcontract  hereunder  for  installation,  opera¬ 
tion,  maintenance  or  training  purposes; 

(viii)  technical  data  or  computer  software  which  is 
in  the  public  domain,  or  has  been  or  is  normally 
released  or  disclosed  by  the  Contractor  or  subcon¬ 
tractor  without  restriction  on  further  disclosure;  and 

(ix)  technical  data  or  computer  software  listed 
or  described  in  an  agreement  incorporated  into  the 
schedule  of  this  contract  which  the  parties  have 
predetermined,  on  the  basis  of  subparagraphs  (i) 
through  (viii)  above,  and  agreed  will  be  furnished 
with  unlimited  rights  [10« 7s 104-9] . 

The  basic  data  clause  is  found  in  DAR  7-104.9, 

Rights  in  Technical  Data  and  Computer  Software.  With  minor 

exceptions,  it  is  used  in  every  contract  in  which  technical 

data  are  specified  to  be  delivered.  The  government's  rights 

to  data  furnished  under  the  contract  are  established  by  the 

clause  by  describing  the  categories  of  data  in  which  the 

government  acquires  limited  and  unlimited  rights.  Any 

specific  data  required  must  be  set  forth  in  the  contract 

schedule.  The  use  of  the  basic  data  clause  in  competitive 
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negotiations  in  which  the  contractor  perceives  the  data  to 
be  proprietary  may  be  a  significant  problem  in  view  of  cur¬ 
rent  directives  aimed  at  increased  competition. 

The  Office  of  Management  and  Budget  has  expressed 
through  Circular  No.  A-109  an  objective  to  "depend  on,  when¬ 
ever  economically  beneficial,  competition  between  similar 
or  differing  system  design  concepts  throughout  the  entire 
acquisition  process  This  goal  is  leading  to  complex 

parallel-development  programs  exploring  various  design  con¬ 
cepts,  e.g.,  Air-Launched  Cruise  Missile  (ALCM)  program  and 
the  Cruise  Missile  Carrier  Aircraft  (CMCA)  program.  The 
government  desires  to  maintain  the  creative  contribution  of 
private  industry  throughout  development  while  releasing 
enough  design  information  to  satisfy  other  contractors  per¬ 
forming  on  related  subsystem  contracts.  This  dichotomy  is 
going  to  require  a  delicate  balance  between  limited  govern¬ 
ment  release  of  "source  selection  sensitive  data"  and  pro¬ 
tection  of  the  contractor's  competitive  position  while  main¬ 
taining  an  efficient  negotiation  process. 

The  government  is  eager  to  preserve  competitive 
options  and  also  to  obtain  full  rights  to  use  what  it  has 
paid  for.  This  results  in  the  creation  of  "source  selection 
sensitive  data, "  defined  as  data,  obtained  by  the  government 
with  full  rights,  which  contain  a  unique  design  concept  that 
must  be  protected  prior  to  the  source  selection  process  in 
order  to  provide  differing  alternatives  under  the  guidance 
of  OMB  Circular  No.  A-109 . 
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Every  major  defense  weapon  system  with  development 
costs  expected  to  exceed  $25  million  or  production  costs  ex¬ 
ceeding  $100  million  must  utilize  formal  source  selection 
procedures.  The  DQD  defines  "source  selection"  as 

the  process  wherein  the  requirements,  facts, 
recommendations,  and  Government  policy  relevant  to 
an  award  decision  in  a  competitive  procurement  of 
a  system  are  examined  and  the  decision  is  made 
[5«259]. 

A  formal  evaluation  of  proposals  submitted  by  interested 
contractors  is  conducted.  The  primary  objective  of  the 
source  selection  process  is 

to  assure  impartial,  equitable,  and  comprehen¬ 
sive  evaluation,  resulting  in  the  selection  of 
that  contractor  whose  proposal  offers  optimum 
satisfaction  of  the  Government’s  cost,  schedule, 
and  performance  objectives  for  the  system  [5*259]. 

Without  protection  of  source  selection  sensitive  data,  the 
government  could,  after  a  lengthy  parallel  development'  pro¬ 
gram,  be  evaluating  proposals  that  have  lost  their  unique 
qualities  due  to  technical  transfusion  and/or  leveling. 

The  purpose  of  this  study,  therefore,  is  to  in¬ 
vestigate  the  relationship  between  government  ownership  of 
source  selection  sensitive  data,  the  level  of  competition, 
and  the  time/cost  of  the  negotiation  process.  This  inves¬ 
tigation  will  attempt  to  meet  three  (3)  major  objectives  by 
answering  eight  (8)  research  questions ^ 

Assum-ptrons 

The  incentives  and/or  protection  required  for  source 
selection  sensitive  data  is  a  problem  which  is  directly 
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related  to  and  determined  by  the  contractor's  perception  of 
the  sensitivity  of  his  data.  Whether  or  not  the  government 
position  agrees  with  the  contractor's  perception,  the  problem 
of  an  adverse  effect  upon  the  level  of  competition  and  the 
time/cost  of  negotiations  may  exist.  While  it  is  recog¬ 
nized  that  various  contractors  may  view  the  sensitivity  of 
their  technical  data  differently,  it  is  not  considered  to  be 
an  issue  that  bears  relevance  to  the  problem  at  hand.  This 
study  assumes  that  a  contractor's  perception  of  the  sensi¬ 
tivity  of  his  data  will  not  change  and  explores  ways  to  de¬ 
crease  the  contractor's  reluctance  to  share  the  data  when 
deemed  necessary. 


Research  Ob.iectives/ 

Research  Questions 

The  primary  objectives  of  this  research  effort  are 
to: 

Objective  1.0  Determine  the  relationships  between 
the  government  ownership  of  source  selection  sensitive  data, 
the  level  of  competition,  the  time/cost  of  the  negotiation 
process,  and  the  level  of  incentives.  To  accomplish  this 
objective,  the  following  research  questions  were  answered: 

1.1  Does  the  current  method  of  government 
acquisition  of  source  selection  sensitive  data 
result  in  decreased  competition,  increased 
time /cost  of  negotiation,  and  inadequate  in- 
centi ves/pr ot ec ti on? 
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1.2  What  factors  are  considered  important 
in  the  contractor’s  decision  to  compete,  and 
how  do  they  relate  to  the  remaining  variables? 

1.3  What  factors  result  in  increased  time/ 
cost  of  negotiation,  and  how  do  they  relate 
to  the  remaining  variables? 

1.4  What  incentives  and/or  protection  are 
currently  offered  to  the  contractor,  and  how 
do  they  relate  to  the  remaining  variables? 

Objective  2.0  Develop  a  decision  model  which  will 
prescribe  the  levels  of  incentives  and/or  protection  which 
the  government  must  utilize  to  assure  alternative  design 
concepts  throughout  the  acquisition  process.  To  accomplish 
this  objective,  the  following  research  questions  were 
answered! 

2.1  What  constitutes  "adequate"  incentives 
and/or  protection  to  assure  alternative  de¬ 
sign  concepts? 

2.2  What  incentives  and/or  protection  can 
the  government  offer  within  the  limitations 
of  the  DAR? 

Objective  3*0  Test  the  use  of  the  model  to  deter¬ 
mine  if  its  use  results  in  a  perceived  increase  in  the  level 
of  competition  and  decrease  in  the  time/cost  of  negotiations. 
To  accomplish  this  objective  the  following  research  questions 
were  answered: 
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3«1  Would  the  use  of  the  incentives  and/or 
protection  represented  by  the  decision  model 
have  resulted  in  greater  assurance  of  alter¬ 
native  design  concepts  through  increased 
competition? 

3.2  Would  the  use  of  the  model  have  de¬ 
creased  the  time  and  cost  of  the  negotiation 
process? 

Scope  and  Limitations 

This  study  will  be  limited  to  contracting  offices 
within  AFSC  and  the  civilian  contractors  with  whom  these 
offices  deal.  Since  AFSC  is  responsible  for  the  first  stage 
of  the  acquisition  process  which  includes  planning,  re¬ 
search,  development,  testing,  and  evaluation,  this  limita¬ 
tion  is  considered  appropriate. 

Justification 

Since  the  inception  of  0MB  Circular  No.  A-109  in 
April  of  1976,  a  trend  toward  more  complex,  interlocking, 
parallel-development  programs  is  being  experienced.  How¬ 
ever,  the  Air  Force  has  not  identified  any  new  procedures, 
strategies,  and  tactics  for  dealing  effectively  with  the 
multitude  of  data  involved.  As  a  result,  some  PCOs  have  had 
difficulty  handling  the  complex  negotiations  of  data  protec¬ 
tion  while  encouraging  increased  competition  (1).  There  are 
no  complete  records  available  which  indicate  the  contractors ’ 
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concerns  and  their  corresponding  reluctance  to  enter  com¬ 
petition  on  weapon  systems  and/or  subsystems.  Neither  is 
there  documented  evidence  of  the  exact  time  and  cost  as¬ 
sociated  with  negotiation  of  such  conflicts  between  con¬ 
tractors  and  the  government.  The  magnitude  of  the  problem 
is  only  evidenced  by  the  increased  requests  for  assistance 
by  the  Office  of  the  Staff  Judge  Advocate  (JAG). 

Therefore,  JAG  has  expressed  concern  over  the  ef¬ 
fectiveness  of  the  current  procedures  for  handling  source 
selection  sensitive  data.  Lieutenant  Colonel  John  A.  Ciucci 
and  Captain  C.  Brandon  Gresham  Jr.,  Office  of  the  Staff 
Judge  Advocate,  AFLC,  have  requested  a  study  be  conducted  to 
determine  the  appropriate  procedures,  strategies,  and  tac¬ 
tics  to  deal  effectively  with  source  selection  sensitive 
data.  Operating  level  personnel  have  expressed  a  concern 
that  the  current  procedures  are  not  adequate.  Captain  Steve 
Lathrop,  former  Contracting  Officer  on  the  Cruise  Missile 
Carrier  Aircraft  (CMCA)  contracts  for  the  Air-Launched 
Cruise  Missile  (ALCM),  stated  that  the  current  procedures 
often  create  delays  in  negotiations  due  to  the  contractors' 
reluctance  to  grant  the  government  substantial  rights  in 
data  (7). 

Literature  Review 

This  research  is  an  exploratory  effort  in  the  area 
of  procedures  and  strategies  for  acquisition  of  the  source 
selection  sensitive  data  involved  in  complex  contract 
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negotiations  involving  related  and  parallel  programs. 

While  no  literature  is  available  in  the  specific  topic  area, 
some  related  data  are  available  in  the  systems  acquisition 
area.  The  Defense  Acquisition  Regulation  (DAR)  contains  the 
procedures  upon  which  the  current  method  of  handling  con¬ 
tractor  data  is  based. 

The  interviews  with  Captain  Gresham  (6)  defined  the 
problem  as  contained  in  the  problem  statement  of  this  study. 
The  complexities  of  the  negotiation  and  source  selection 
processes  involved  in  the  A1CM  contracts  and  the  interre¬ 
lated  CMCA  and  Cruise  Missile  Interface  (CMI)  contracts  were 
discussed  at  length.  Captain  Gresham  stated  that  the  divi¬ 
sions  which  have  surfaced  and  will  continue  to  surface  dur¬ 
ing  the  ALOW/ CMC a/CMI  negotiations  are  representative  of 
problems  associated  not  only  with  a  complex  contractual 
process  as  the  A1CM  but  also  any  contract  associated  with 
parallel  development  or  interlocking  subsystems.  He  also 
said  that  the  present  system  of  handling  source  selection 
sensitive  data  is  to  use  the  standard  clauses  as  contained 
in  the  DAR.  These  standard  clauses  have  not  proven  to  be 
adequate  in  resolving  problems  associated  in  parallel  de¬ 
velopment  and  interlocking  subsystems  contracts  (6). 

The  interview  with  Captain  Lathrop  (7)  showed  that, 
while  the  ALCM  and  related  contracts  were  extremely  complex 
and  involved  many  contractors,  the  lessons  learned  can  be 
applied  to  contracts  much  less  complex.  Future  of  defense 


acquisition  will  contain  more  off-the-shelf  buys  of  com¬ 
mercial  products  or  modifications  of  off-the-shelf  products. 
Coupled  with  the  policy  of  increased  competition  during  all 
phases  of  systems  acquisition,  this  increased  emphasis  on 
off-the-shelf  buys  will  present  contractors  with  the  same 
problems  evidenced  in  the  A1CM  negotiation  process. 

The  remaining  chapters  of  this  thesis  discuss  the 
achievement  of  our  research  objectives.  Chapter  2  — 
Methodology  —  discusses  the  methods  which  were  used  for 
data  collection  and  analysis.  Chapter  3  —  Data  Summary  and 
Analysis  —  presents  the  results  of  the  interviews  conducted 
and  the  results  of  the  statistical  tests  performed.  Chapter 
4  —  Findings  and  Recommendations  —  discusses  our  proposed 
model  based  upon  interview  results  and  provides  suggestions 
for  further  study. 


Chapter  2 


METHODOLOGY 

This  chapter  describes  the  universe,  population, 
and  sample  from  which  data  were  collected  for  this  research, 
the  techniques  employed  in  collecting  the  data,  the  statis¬ 
tical  tests  used  in  the  analysis  of  the  data,  the  develop¬ 
ment  of  the  model,  and  research  assumptions  and  limitations. 

Description  of  Universe, 

Population  and  Sample 

The  universe  for  this  research  project  consists  of 
all  major  weapons  system  contracts  within  the  Air  Force. 

This  includes  not  only  contracts  involving  source  selection 
sensitive  data  such  as  parallel  development  programs,  e.g., 
the  group  of  8  ALCM  contracts,  and  contracts  which  include 
modification  of  a  contractor's  design,  e.g.,  the  KC-10  con¬ 
tract,  but  also  those  contracts  in  which  source  selection 
sensitive  data  were  not  involved. 

The  population  of  interest  within  the  specified 
universe  originally  was  designed  to  include  those  AFSC  con¬ 
tracts  which  satisfy  both  of  the  following  criteriat 

(1)  the  group  of  contracts  associated  with  a  com¬ 
petitive  effort,  and 

(2)  the  group  of  contracts  in  which  time  and  cost  of 
negotiation  effort  can  be  measured. 


The  contracts  associated  with  a  competitive  effort  are  de¬ 
fined  as  not  only  those  which  involved  more  than  one  pro¬ 
posal  at  the  evaluation  phase  of  the  acquisition  process, 
but  also  those  contracts  which  were  related  to  parallel  de¬ 
velopment  programs  in  competition  for  desirable  follow-on 
contracts . 

Established  weapons  systems  program  offices  (SPOs) 
often  do  not  retain  any  memory  of  the  initial  acquisition 
process  and  the  details  of  the  negotiation  effort.  Many 
of  them  were  established  prior  to  the  emphasis  upon  in¬ 
creased  competition  as  established  by  0MB  Circular  No.  A-109. 
Consequently,  an  attempt  was  made  to  limit  the  primary 
population  to  those  contracts  within  AFSC  for  which  the 
necessary  information  with  regard  to  the  level  of  competi¬ 
tion  and  the  time/cost  of  the  negotiation  effort  is  available. 

A  deliberate  sample  of  20  contracts  from  the  primary 
population  was  taken.  The  sample  was  chosen  for  both  the 
availability  of  information  and  the  belief  of  the  JAG  of¬ 
fice  that  this  group  of  contracts  included  a  valid  cross- 
section  of  some  typical  programs  involving  the  acquisition 
of  data.  The  criteria  for  choosing  the  particular  contracts 
under  consideration  include d« 

(l)  ten  contracts  which  involved  parallel  develop¬ 
ment  programs  or  follow-on  competition  in  which  contractors 
were  required  to  share  data  which  could  adversely  affect 
their  competitive  position;  and 


(2)  ten  contracts  of  comparable  value  in  which  no 
source  selection  sensitive  data  was  involved. 

In  addition,  the  JAG  office  chose  contracts  for  which 
information  and  government  contracting  personnel  were 
available . 

The  prime  objective  in  sampling  was  to  select  a 
group  of  contracts  which  would  provide  us  with  available 
data  in  order  to  determine  the  relationships  between  the 
acquisition  of  source  selection  sensitive  data  and  the  level 
of  competition  and/or  time  and  cost  of  the  negotiation  proc¬ 
ess.  The  sample  was  also  chosen  to  provide  a  cross-section 
of  major  weapons  system  contractors.  While  we  recognize 
that  a  purposive  non-random  sample  limits  the  confidence 
with  which  we  can  make  inferences  to  the  population  of  con¬ 
cern,  the  sample  was  chosen  with  the  expert  advice  of  the 
JAG  office  and  the  intent  to  provide  a  reasonable  cross- 
section  of  situations,  PCOs,  and  contractors  in  order  to 
build  meaningful  theory  for  further  research. 

Data  Collection 

Data  collection  was  performed  in  two  phases.  The 
first  phase  involved  the  identification  and  definition  of 
the  factors  important  in  answering  the  research  questions 
associated  with  research  objectives  1.0  and  2.0.  The  second 
phase  involved  verification  of  a  model  by  answering  the 
research  questions  associated  with  research  objective  3*0. 
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Phase  I.  The  first  step  was  identification  and  de¬ 
finition  of  the  factors  involved  in  the  analysis  of  the 
impact  of  source  selection  sensitive  data  on  the  acquisi¬ 
tion  process.  The  data  were  obtained  through  a  survey  of 
PCOs  and  contractors  involved  in  the  negotiation  of  the  con¬ 
tracts  chosen  for  the  sample.  The  survey  was  conducted  by 
using  structured  personal  and  telephone  interviews.  While 
some  of  the  data  could  have  been  obtained  by  abstractions  of 
information  from  the  contract  files,  much  of  the  data  re¬ 
quired  insight  into  the  perception  of  the  individuals  in¬ 
volved  in  the  negotiation  process.  Therefore,  the  use  of 
interviews  added  the  benefit  of  obtaining  obscure  data  from 
people  knowledgeable  in  the  area.  The  interview  technique 
also  allowed  for  the  use  of  some  questions  of  interest  which 
were  open-ended  and  subjective.  These  questions  solicited 
judgments  and  opinions  from  the  interviewees.  In  addition, 
some  of  the  terms  and  concepts  utilized  were  vague  and  am¬ 
biguous  and  required  discussion  with  the  interviewees  in 
order  to  clarify  the  operational  definitions. 

While  personal  interviewing  provided  the  advantages 
of  "depth  and  detail  of  information"  and  "far  exceeds,  in 
volume  and  quality,  the  information  we  can  usually  secure 
from  telephone  and  mail  surveys,"  both  personal  and  tele¬ 
phone  interviews  were  conducted  [3*268}.  Due  to  the  wide 
geographic  dispersion  of  the  sample  interviewees,  personal 
interviews  were  not  possible  with  all  of  the  PCOs  and 
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contractors  due  to  the  limitation  on  time  and  funds.  Con¬ 
sequently,  personal  interviews  were  conducted  with  individ¬ 
uals  located  at  Wright-Patterson  AFB,  and  telephone  inter¬ 
views  were  conducted  with  those  people  located  elsewhere. 

In  addition  to  cost,  bias  is  one  of  the  two  major 
problems  encountered  in  personal  interviewing.  A  number  of 
studies  have  indicated  that  "an  interviewer  can  distort  the 
results  of  any  survey  by  inappropriate  suggestions,  word 
emphasis,  tone  of  voice,  and  question  rephrasing  [3*277]." 
The  presence  of  a  personal  interviewer  was  not  considered 
to  be  a  problem  due  to  the  professional  nature  of  the  ques¬ 
tions  and  the  consistent  use  of  the  same  interview  guides 
for  both  personal  and  telephone  interviews  (See  Appendices 
C  and  D) . 

The  interview  guides  included  a  cover  letter  de¬ 
signed  to  introduce  and  explain  the  research  effort  and  a 
list  of  questions  designed  to  identify  pertinent  background 
information  on  the  sample  contracts.  This  information  in¬ 
cluded  data  on  the  level  of  competition,  the  length  of  the 
negotiation  effort,  and  the  cost  of  contracts  involving 
the  acquisition  of  source  selection  sensitive  data  as  com¬ 
pared  to  those  contracts  in  which  no  source  selection  sensi¬ 
tive  data  were  acquired. 

Recommendations  for  improved  procedures  were  solic¬ 
ited  when  warranted  by  the  responses  to  the  questions  listed 


in  the  interview  guides.  The  requests  for  recommendations 
focused  primarily  on  identifying  potential  contractual  in¬ 
struments  which  would  provide  incentives  and  protection  to 
the  contractor  to  alleviate  any  perceived  problems. 

The  primary  purpose  of  the  interviews  was  to  provide 
the  data  necessary  to  answer  the  research  questions  asso¬ 
ciated  with  research  objectives  1.0  and  2.0. 

Objective  1.0.  The  interviews  with  PCOs  were  de¬ 
signed  to  provide  data  on  the  level  of  competition,  the  time 
and  cost  of  the  negotiation,  and  whether  incentives  were  in¬ 
volved  in  each  of  the  contracts  sampled.  The  "level  of 
competition"  was  measured  by  the  number  of  offers  received 
in  response  to  the  RFP.  The  "time  of  negotiation  effort"  was 
measured  in  days  from  receipt  of  the  purchase  request 
through  contract  award.  The  "cost  of  the  negotiation"  was 
measured  in  dollars  by  comparing  the  engineering  estimate  of 
the  contract  cost  included  in  the  purchase  request  with  the 
actual  negotiated  cost.  The  "incentives"  were  measured  by 
the  inclusion  or  omission  of  a  specific  clause  for  protec¬ 
tion  of  data  rights  in  the  contractual  document. 

The  data  collected  were  to  be  compared  to  the  aver¬ 
age  number  of  competitors  and  length  of  the  average  acquisi¬ 
tion  cycle  for  similar  negotiated  contracts  in  order  to 
answer  research  question  1.1.  The  interviews  with  PCOs, 
however,  showed  that  there  was  no  average  length  of  an 
acquisition  cycle  for  major  weapon  systems  against  which  to 
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compare  the  data.  Only  one  PCO  felt  that  his  program  had 
taken  longer  to  negotiate  than  what  he  considered  to  be  a 
normal  time.  The  additional  time  required  was  not  attrib¬ 
utable  to  the  negotiation  process,  but  was  due  to  delays  in 
Defense  System  Acquisition  Review  Council  (DSARC)  action. 
Since  most  PCOs  felt  that  their  negotiation  process  re¬ 
quired  a  "normal"  period  of  time  and  written  records  are 
not  maintained  to  reflect  the  length  of  the  negotiation  ef¬ 
fort,  the  data  collected  to  represent  the  "time  of  negotia¬ 
tion  effort"  were  found  to  be  sketchy  and  subjective. 
Specific  answers  are  addressed  in  Chapter  3. 

In  addition,  the  contractor  was  queried  with  regard 
to  whether  or  not  the  inclusion  of  source  selection  sensi¬ 
tive  data  in  the  contract  resulted  in  a  higher  cost  in 
order  to  provide  some  financial  protection. 

The  interviews  with  PCOs  provided  information  with 
regard  to  the  number  of  contractors  solicited  and  their 
identity  in  order  to  ascertain  the  reasons  for  some  not  sub 
mitting  offers.  Along  with  this  information,  interviews 
with  contractors  provided  the  factors  considered  important 
in  the  decision  to  compete  in  order  to  answer  research  ques 
tion  1.2.  In  addition  both  PCOs  and  contractors  were  asked 
to  relate  factors  which  they  perceived  affected  the  time 
and  cost  of  the  contract  negotiations  to  which  they  were  a 
party.  This  data  would  provide  the  answer  for  research 
question  1.3.  Finally,  both  groups  were  asked  to  discuss 


their  perception  of  how  incentives  and/or  protection  re¬ 
lated  to  the  other  variables  of  government  ownership  of 
source  selection  sensitive  data,  the  time/cost  of  negotia¬ 
tions,  and  the  level  of  competition.  Data  acquired  in  this 
process  were  used  to  answer  research  question  1.4. 

Objective  2.0.  In  order  to  answer  research  question 
2.1,  the  interviews  with  contractors  required  that  they  de¬ 
fine  their  perception  of  "adequate"  incentives  and/or  pro¬ 
tection  when  source  selection  sensitive  data  are  involved. 
PCOs  were  asked  to  define  the  limits  of  incentives  and  pro¬ 
tection  which  the  government  can  offer  to  contractors  in 
negotiations  where  source  selection  sensitive  data  were  in¬ 
volved.  Contracting  officers  were  also  asked  whether  the 
incentives  and  protection  which  they  offered  to  contractors 
were  sufficient  to  insure  alternate  design  concepts  through¬ 
out  the  acquisition  process.  The  information  gathered  in 
this  last  series  of  interviews  provided  the  answer  to  re¬ 
search  question  2.2. 

The  key  to  acquiring  the  data  necessary  to  answer 
research  questions  1.1  through  2.2  was  the  quality  and  accu¬ 
racy  of  the  interview  guide.  The  interview  guide  was  tested 
prior  to  its  use.  A  test  ensures  that 

.  .  .  the  questions  meet  the  objectives  of 
the  survey;  all  important  phases  of  the  survey 
have  been  adequately  covered;  the  questions  stim¬ 
ulate  respondent  cooperation;  the  questions  are 
in  satisfactory  order;  and  the  questions  are 
completely  understood  by  the  respondents  [4i2-3]. 
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The  test  was  conducted  at  the  Air  Force  Institute  of 
Technology  (AFIT)  with  contracting  and  acquisition  instruc¬ 
tors  from  the  graduate  and  continuing  education  programs. 
After  a  personal  interview,  the  questions  were  discussed 
with  the  interviewees  in  order  to  "determine  what  their 
feelings,  confusions,  and  reactions  are  to  the  questioning 
process  and  to  specific  questions  [3s 207]."  The  AFIT  in¬ 
structors  are  familiar  with  both  contracting  policies  and 
procedures  and  proper  research  techniques.  Therefore,  the 
AFIT  instructors  were  considered  to  be  a  valid  test  group. 
Changes  which  were  recommended  by  the  test  group  were  con¬ 
sidered  for  revision  of  the  interview  guide. 

In  order  to  obtain  the  support  and  cooperation  of 
the  interviewees,  the  interviewers  provided  all  interviewees 
with  advanced  notice  of  the  interview  and  established  a 
convenient  time  for  the  interview. 

The  data  generated  by  the  interviews  were  analyzed 
in  order  to  determine  the  relationships  between  the  govern¬ 
ment  acquisition  of  source  selection  sensitive  data,  the 
level  of  competition,  the  time/cost  of  the  negotiation  proc¬ 
ess,  and  levels  of  incentives  and  protection.  Data  revealed 
that  a  relationship  does  exist  between  the  government  acqui¬ 
sition  of  source  selection  sensitive  data  and  increased  cost 
for  contract  negotiation.  The  level  of  competition  appeared 
to  be  unaffected,  and  data  results  for  the  time  relationship 
were  inconclusive.  The  information  obtained  in  Phase  I  was 


utilized  to  develop  a  decision  model  which  was  verified 
in  Phase  II  of  data  analysis. 


Phase  II.  Phase  II  involved  the  verification  of 
the  decision  model  derived  from  analysis  of  the  results  of 
Phase  I.  Information  concerning  the  effectiveness  of  the 
decision  model  was  solicited  from  the  PGOs  and  contractors 
involved  in  those  contracts  investigated  in  Phase  I  which 
included  acquisition  of  source  selection  sensitive  data. 

Objective  3.0.  The  ten  sample  contracts  including 
source  selection  sensitive  data  were  reconsidered  by  the 
PCOs  and  contractors  in  order  to  determine  if  utilization  of 
the  decision  model  would  have  resulted  in  greater  possibil¬ 
ity  of  alternate  design  concepts  which  would  have  increased 
the  level  of  competition.  This  portion  of  the  interviews 
was  used  to  answer  research  question  3*1 •  In  order  to  an¬ 
swer  research  question  3.2,  PCOs  and  contractors  were  asked 
if  use  of  the  decision  model  would  have  resulted  in  de¬ 
creased  time  and  cost  of  the  negotiation  process  for  the 
contracts  with  which  they  were  involved. 

Data  Analysis 

Data  analysis  was  performed  in  two  phases  with  each 
phase  corresponding  to  a  data  collection  phase. 

Phase  I.  Objective  1.0.  The  information  which  was 
obtained  during  the  first  phase  of  interviews  was  analyzed 
to  determine  if  relationships  exist  between  the  government 


acquisition  of  source  selection  sensitive  data,  the  level  of 
competition,  the  time/cost  of  negotiations,  and  the  level  of 
incentives  and/or  protection.  The  data  were  examined  in  two 
groups t 

(1)  Contracts  in  which  source  selection  sensitive 
data  were  not  involved,  and 

(2)  Contracts  in  which  acquisition  of  source  selec¬ 
tion  sensitive  data  was  an  issue. 

The  mean  time  and  cost  of  negotiations  and  the  mean  number 
of  offers  were  calculated  for  each  group.  The  difference 
between  the  means  was  tested  to  determine  if  the  perceived 
sensitivity  of  data  involved  in  the  acquisition  process 
affected  the  level  of  competition  and/or  the  time  and  cost 
of  negotiations.  The  test  was  performed  using  an  SPSS 
T-TEST  program. 

The  T-TEST  is  a  measure  of  the  value  of  the  popula¬ 
tion  mean  (j,.  In  this  study,  the  T-TEST  was  used  to  deter¬ 
mine  if  a  difference  exists  between  the  means  in  the  two 
groups  as  measured  by  the  interviewers.  The  hypothesis 
tested  was« 

Hqj  ^2~^1  =  0?  No  rela‘ti°nsh^-P  exists 

Hjt  d2-(il  ^  °!  Rela'tionsl^-P  does  exist. 

The  90  percent  confidence  level  was  used  for  the 
T-TEST.  Therefore,  through  the  use  of  this  test,  a  90  per¬ 
cent  confidence  interval  was  calculated  which  permitted  the 


rejection  of  the  null  hypothesis.  This  rejection  provided 
a  valid  basis  for  further  study  in  Phase  II  of  the  research 
effort.  Detailed  analysis  is  contained  in  Chapter  3. 

Objective  2.0.  The  data  acquired  in  Riase  I  were 
also  analyzed  to  assist  in  the  development  of  a  decision 
model.  The  contractors'  definitions  of  "adequate"  incen¬ 
tives  and  the  limits  of  incentives  and  protection  which  the 
government  can  offer  to  contractors  were  considered  in  the 
construction  of  the  decision  model.  Due  to  the  subjective 
nature  of  question  fifteen  of  the  PCO  interview  guide  and 
question  eight  of  the  contractor's  interview  guide,  the  de¬ 
cision  model  was  developed  consistent  with  a  concensus  of 
the  recommendations  obtained  during  interviews. 

Phase  II.  Objective  3.0.  Data  collected  in  the 
second  phase  were  analyzed  to  determine  whether  or  not  the 
use  of  the  decision  model  resulted  in  increased  competition 
and/or  decreased  time  and  cost  of  negotiations.  The  dif¬ 
ferences  were  determined  through  comparison  of  the  actual 
levels  of  competition,  time  and  cost  of  negotiation,  and 
levels  of  incentives  and/or  protection  of  the  contracts 
sampled  with  the  perceived  values  of  those  same  variables 
which  would  have  been  experienced  by  using  the  model.  Re¬ 
sults  discussed  in  Chapter  3  indicate  that  use  of  the  model 
would  have  provided  incentives  and  protection  resulting  in 
decreased  time  and  cost  of  negotiations  for  seventy  percent 
of  the  contractors  interviewed. 
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Assumptions 

(1)  The  deliberate  sample  was  representative  of  the 
typical  programs  involving  the  acquisition  of  data. 

(2)  The  T-TEST  was  appropriate  for  the  type  and 
nature  of  the  data  obtained. 

(3)  Interviewers  provided  honest  answers  based  upon 
valid  opinions  and  information. 

Limitations 

(1)  Time  and  resources  available  to  the  researchers. 

(2)  Time  available  to  interviewees. 

(3)  Access  to  contracts  for  major  weapons  systems  in 
which  source  selection  sensitive  data  were  an  issue  and  con¬ 
tracts  in  which  they  were  not  for  comparison  purposes. 
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Chapter  3 


DATA  SUMMARY  AND  ANALYSIS 

This  chapter  discusses  the  data  collected  and  the 
analysis  of  the  data.  This  discussion  includes  a  considera¬ 
tion  of  the  interview  questions,  a  statistical  and  descrip¬ 
tive  summary  of  the  interview  results,  and  an  analysis  of 
the  statistical  tests  performed  on  the  interview  results. 

Our  findings  and  recommendations  based  upon  those  results 
are  contained  in  Chapter  4. 

Interview  Procedures 

Interviews  were  conducted  with  PCOs  and  industry 
contracting  personnel  in  accordance  with  the  research  design 
outlined  in  Chapter  2.  Twenty  interviews  were  conducted 
with  PCOs  corresponding  to  the  20  contracts  included  in  the 
deliberate  sample.  Ten  interviews  were  conducted  with  in¬ 
dustry  personnel  representing  10  major  weapons  system  con¬ 
tractors.  As  mentioned  earlier,  JAG  selected  the  20  con¬ 
tracts  to  reflect  a  reasonable  cross-section  of  situations, 
PCOs,  and  contractors. 

The  interviewees  were  extremely  helpful  and  coopera¬ 
tive  and,  wherever  possible,  made  their  historical  files  and 
records  available  to  us  in  order  to  obtain  precise  dates  and 
cost  figures.  Sixty  percent  of  the  interviews  were  personal 
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interviews,  and  the  remaining  forty  percent  were  telephone 
interviews.  On  the  average,  the  interviews  lasted  approx¬ 
imately  i*>5  minutes.  Despite  busy  schedules,  all  of  the  in- 
terviewees  were  willing  to  sperfd  the  time  necessary  to  an¬ 
swer  all  of  the  interview  questions. 

Interview  Questions  -  PCOs 

The  interviews  with  contracting  officers  consisted 
of  15  basic  questions  as  outlined  in  the  interview  guide 
(Appendix  C) .  The  questions  were  asked  in  the  order  shown 
on  the  guide  deviating  only  to  clarify  responses  and  issues 
as  necessary.  The  responses  were  summarized  by  question  to 
determine  relationships  and  answers  to  specific  research 
questions  associated  with  our  research  objectives. 

Question  #  1.  How  many  offers  did  you  solicit  on  this  pro¬ 
curement  and  to  which  contractors  were  RFPs  issued? 

In  order  to  clarify  the  competitive  nature  of  the 
contracts,  each  interview  was  begun  with  a  discussion  of  the 
number  of  contractors  solicited.  The  contractors  were  iden¬ 
tified  to  aid  in  compiling  the  list  of  industry  interviewees. 
In  all  cases,  triis  information  was  readily  available. 

Question  #  2.  How  many  offers  did  you  receive? 

The  second  question  dealt  with  the  number  of  offers 
received  as  a  result  of  the  solicitation.  Parallel  develop¬ 
ment  contracts  were  also  identified  with  this  question. 
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The  responses  to  these  first  two  questions  were  tabulated 
into  two  groups  representing  those  contracts  which  did  not 
involve  source  selection  sensitive  data  and  those  contracts 
in  which  the  acquisition  of  source  selection  sensitive  data 
was  an  issue.  A  comparison  of  the  two  groups  sought  to  de¬ 
termine  any  existing  relationship  between  the  government 
ownership  of  source  selection  sensitive  data  and  the  level 
of  competition  in  response  to  research  objective  1.0. 
(Statistical  analyses  appear  in  the  data  analysis  section  of 
this  chapter) .  We  were  unable  to  support  the  contention 
that  the  current  method  of  government  acquisition  of  source 
selection  sensitive  data  results  in  decreased  competition. 

In  answer  to  research  question  1.2,  ninety-five  percent  of 
the  PCOs  indicated  that  major  weapons  system  contractors 
were  eager  to  win  large  government  contracts  and  did  not 
appear  to  be  reluctant  to  submit  proposals  in  response  to 
the  solicitations.  If  the  contractor  was  concerned  about 
specific  data,  the  proposal  would  include  a  request  to  pro¬ 
tect  the  data  by  granting  the  government  only  limited 
rights.  This  issue  would  then  be  addressed  at  some  later 
date  in  discussions  between  the  contractor  and  the  govern¬ 
ment. 

Another  consideration  brought  to  our  attention  by 
the  PCOs  was  the  fact  that  only  a  limited  number  of  con¬ 
tractors  are  qualified  for  solicitation  on  major  weapons 
system  RFPs,  resulting  in  a  mailing  list  which  includes  two 
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or  three  contractors.  Due  to  the  public  nature  of  government 
contracts,  these  contractors  have  been  preparing  for  release 
of  this  RFP  for  months  or,  in  some  cases,  for  years.  In 
several  instances,  all  of  the  contractors  solicited  re¬ 
sponded  with  a  proposal.  Due  to  the  size  of  the  potential 
contract  and  the  infrequency  of  major  weapons  system  buys, 
the  contractors  are  compelled  to  present  a  proposal  to  the 
government  whenever  the  opportunity  arises. 

Question  #  3.  How  many  days. elapsed  between  receipt  of  pur¬ 
chase  request  and  contract  award  in  this  procurement? 

This  question  was  designed  to  answer  research  ques¬ 
tion  1.3  and  to  measure  the  precise  number  of  days  required 
for  the  acquisition  process  on  the  contract  under  discussion. 
The  acquisition  process,  for  the  purposes  of  this  research 
effort,  was  defined  as  the  period  beginning  with  receipt  of 
the  purchase  request  and  ending  with  contract  award.  The 
date  utilized  for  contract  award  was  the  date  on  the  Certi¬ 
ficate  of  Current  Cost  and  Pricing  Data.  We  found  that  in 
most  cases  these  data  were  either  unavailable,  despite  per¬ 
sonal  interviews  with  the  government  contracting  personnel 
who  had  been  directly  involved,  or  the  dates  were  so  af¬ 
fected  by  a  variety  of  events  that  any  results  would  be  in¬ 
conclusive.  In  major  weapons  system  acquisition,  some  of 
these  events  include  formal  source  selection  procedures, 

DSARC  reviews,  and  budgetary  delays.  In  many  cases,  there 
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was  no  purchase  request,  and  no  record  was  maintained  of  a 
precise  date  for  initiation  of  the  acquisition  process. 

The  first  recorded  date  was  often  the  release  date  for  the 
RFP.  Likewise,  no  record  was  maintained  to  reflect  actual 
negotiations  with  the  contractors.  PCOs  indicated  that,  in 
the  case  of  parallel  development  programs.,  negotiations  were 
continuous  from  receipt  of  the  proposal  to  contract  award. 

Question  #  4.  What  is  the  average  length  of  negotiation 
time  for  similar  contracts? 

The  fourth  question  attempted  to  establish  an  aver¬ 
age  length  of  time  for  negotiation  of  similar  contracts. 

The  difference  between  the  answers  to  this  question  and  the 
third  question  would  then  indicate  any  relationships  that 
might  exist  between  the  government  acquisition  of  source 
selection  sensitive  data  and  the  length  of  time  required  for 
contract  negotiation.  However,  the  lack  of  information  to 
answer  the  previous  question  coupled  with  the  PCO's  in¬ 
ability  or  reluctance  to  specify  an  average  negotiation 
time  made  it  impossible  to  conduct  any  meaningful  compari¬ 
son.  Consequently,  we  were  also  unable  to  support  the  con¬ 
tention  that  the  current  method  of  acquiring  source  selec¬ 
tion  sensitive  data  results  in  an  increase  in  the  length  of 
the  negotiation  process.  On  the  other  hand,  we  were  also 
unable  to  conclude  that  the  time  associated  with  negotiation 
is  unaffected.  Uhless  the  necessary  information  is  captured 


on  future  acquisitions,  it  will  be  difficult  to  answer  this 
research  question  with  any  significant  degree  of  certainty 
or  confidence . 

Six  PCO  interviews  did  suggest  that  negotiation  of  a 
data  rights  clause  was  a  time-consuming  element  in  the  over¬ 
all  negotiation  process.  In  one  case,  a  PCO  indicated  that 
ten  days  of  valuable  negotiation  time  had  been  lost  while 
the  government  team  and  the  contractor  team  quibbled  over 
the  meaning  of  one  word  in  a  tailored  special  provision 
dealing  with  data  rights.  In  another  case,  disagreements 
over  data  rights  and  other  technicalities  resulted  in  the 
negotiation  of  a  complex  interface  document  detailing  pre¬ 
cautionary  measures  designed  to  protect  source  selectj.cn 
sensitive  data  until  a  source  decision  was  made.  It  ap¬ 
pears  that  the  associated  variance  in  the  distribution  of 
time  is  so  wide  that  the  average  is  not  necessarily  a  mean¬ 
ingful  point  estimator  of  time. 

Question  #  5.  What  was  the  engineering  estimate  of  the  con¬ 
tract  cost  contained  in  the  purchase  request? 

Another  relationship  sought  by  research  objective 
1.0  was  between  the  government  ownership  of  source  selection 
sensitive  data  and  the  cost  arrived  at  as  a  result  of  the 
negotiation  process.  This  question  was  designed  to  provide 
the  government  engineering  estimate  for  the  effort  called 
for  by  the  RFP.  These  estimates  are  generally  based  upon 
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engineering  expertise  and  recent  experience  on  similar  con¬ 
tracts  often  with  the  same  contractors  being  solicited.  In 
most  cases,  we  were  able  to  obtain  the  information  sought 
by  the  question.  In  two  instances,  the  programs  were  so  de¬ 
pendent  upon  budget  considerations  that  the  required  effort 
was  dictated  by  the  funds  available  at  the  time  of  negotia¬ 
tion.  These  programs  did  not  attempt  to  establish  an  engi¬ 
neering  estimate  early  in  the  program.  The  responses  to 
this  question  were  then  compared  to  the  responses  to  ques¬ 
tion  #  6  to  determine  if  a  relationship  exists. 

Question  #  6.  What  was  the  actual  negotiated  contract  cost? 

The  actual  negotiated  contract  cost  is  a  matter  of 
public  record  and  was  available  in  all  cases.  The  data  col¬ 
lected  as  a  result  of  the  fifth  and  sixth  questions  were 
divided  into  groups  in  accordance  with  the  research  design 
outlined  in  Chapter  2.  The  difference  between  the  engi¬ 
neering  estimate  and  the  actual  negotiated  cost  was  cal¬ 
culated  in  each  case  both  for  the  contracts  in  the  group  in¬ 
volving  the  acquisition  of  source  selection  sensitive  data 
and  for  the  contracts  in  the  group  in  which  source  selection 
sensitive  data  were  not  an  issue.  The  calculated  differ¬ 
ences  were  compared,  and  a  relationship  was  recognized  in¬ 
dicating  that  government  ownership  of  source  selection  sen¬ 
sitive  data  does  tend  to  result  in  increased  program  costs 
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after  negotiation.  The  statistical  analysis  to  support  this 
hypothesis  appears  in  the  data  analysis  section  of  this 
chapter . 

In  answer  to  research  question  1.3,  the  interviews 
with  PCOs  indicated  that,  in  the  case  of  parallel  develop¬ 
ment  programs,  complex  interface  documents  cost  money  to 
draft  and  to  implement.  In  one  example,  a  PCO  cited  a  con¬ 
tract  in  which  29  extra  people  were  required  in  order  to 
assure  that  certain  key  people  would  not  have  access  to  both 
contractors'  technical  data  thereby  increasing  the  likeli¬ 
hood  of  inadvertent  leaks  and  technical  transfusion  prior  to 
source  selection. 

Competitive  programs  in  which  source  selection  sen¬ 
sitive  data  were  not  an  issue  followed  a  pattern  resulting 
in  a  contract  award  at  a  lower  price  than  originally  esti¬ 
mated.  On  the  other  hand,  those  negotiations  in  which  data 
protection  was  a  concern  resulted  in  a  higher  price  for  the 
government  than  originally  estimated  in  all  but  one  case. 
Interviews  indicated  that  while  contractors  definitely  want 
the  awards,  they  are  reluctant  to  make  any  major  concessions 
that  would  jeopardize  not  only  their  competitive  positions 
with  respect  to  the  current  source  selection  but  also  with 
respect  to  future  source  selections  and  commercial  ventures. 

Question  #  7.  Of  the  technical  proposals  submitted,  were 
any  identified  as  containing  source  selection  sensitive 
material? 
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To  further  establish  the  validity  of  the  deliberate 
sample,  the  PCOs  were  asked  whether  the  technical  proposals 


submitted  in  response  to  the  subject  RFPs  contained  any 
source  selection  sensitive  material  identified  within  the 
proposal  itself  or  orally  by  the  contractor.  In  every  case, 
the  FCO  verified  the  JAG  groupings  which  established  that 
our  sample  included  10  contracts  with  source  selection  sensi 
tive  data  problems  and  10  contracts  with  no  data  problems. 

Question  #  8.  Have  you  detected  a  reluctance  on  the  part  of 
contractors  to  submit  technical  proposals  containing  their 
state-of-the-art  technology? 

This  question  was  of  a  general  nature  calling  upon 
the  PCO's  experience  in  order  to  establish  whether  contrac¬ 
tors  tend  to  be  reluctant  to  submit  technical  proposals  con¬ 
taining  their  state-of-the-art  technology.  Sixty-five  per¬ 
cent  of  the  PCOs  interviewed  have  detected  a  reluctance  on 
the  part  of  contractors  to  share  this  information  with  the 
government  in  their  proposals.  One  PCO  indicated  that  this 
problem  does  not  only  apply  to  prime  contractors  but  to  sub¬ 
contractors  as  well.  Some  subcontractors  have  even  refused 
to  bid  because  of  the  flowdown  of  contractual  terms  and  con¬ 
ditions  referring  to  unlimited  rights  in  technical  data. 
Limited  rights  clauses  are  not  frequently  accepted  by  major 
weapons  system  PCOs.  One  PCO  even  enlists  the  aid  of  the 
Administrative  Contracting  Officer  (ACO)  and  the  Defense 
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Contract  Audit  Agency  (DCAA)  to  collect  concrete  proof  of 
money  spent  at  private  expense  in  development  of  specific 
technology  before  allowing  a  limited  rights  clause  in  his 
contract. 

The  PCOs  who  had  experienced  contractor  reluctance 
tried  to  clarify  technology  questions  during  negotiations 
but  were  often  faced  with  evasive  answers.  One  contractor 
even  openly  stated  that  it  had  no  intention  to  share  its 
proprietary  information  regarding  technological  advances. 

The  PCOs  who  indicated  that  they  did  not  detect  any 
contractor  reluctance  generally  qualified  their  answers  with 
the  statement,  "they  don't  like  it,  but  they  want  the  con¬ 
tract  enough  to  give  us  what  we  ask  for. "  This  in  itself 
indicated  some  degree  of  reluctance.  These  contractors  dif¬ 
fered  from  the  others  when  they  made  a  management  decision 
to  take  a  chance  by  divulging  their  technology  early  in  the 
program . 

Question  #  9.  Have  any  contractors  discussed  with  you  the 
problem  of  disclosure  of  their  technical  proposals  which 
could  contain  state-of-the-art  technology? 

Only  fifty-five  percent  of  the  PCOs  interviewed  have 
been  approached  by  contractors  to  discuss  the  potential  pro¬ 
blem  of  disclosure  of  portions  of  their  technical  proposals 
which  could  contain  state-of-the-art  technology.  One  PCO 
indicated  that,  while  contractors  are  reluctant  to  share 
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their  data  with  the  government,  they  are  not  eager  to  risk 
insulting  the  PCO  by  insinuating  that  he  or  she  might  be 
party  to  charges  of  technical  transfusion.  At  least  one  of 
the  concerned  contractors  had  good  reason  to  be  concerned. 

A  Freedom  of  Information  request  for  cost  data  included  in 
one  of  its  proposals  had  been  granted,  and  the  contractor 
was  fearful  that  technical  data  would  be  next. 

Question  #  10.  Have  you  experienced  difficulty  evaluating 
technical  proposals  in  which  the  contractor  has  apparently 
withheld  some  technical  information? 

In  view  of  the  responses  to  question  #  8,  the  PCOs 
were  asked  if  they  had  experienced  any  difficulty  evalu¬ 
ating  proposals  in  which  the  contractor  had  apparently  with¬ 
held  some  technical  information.  This  did  not  seem  to  pre¬ 
sent  a  major  problem  to  the  PCOs.  Only  twenty-five  percent 
had  sensed  that  technical  questions  remained  unanswered 
after  fact-finding  discussions  had  been  conducted  with  the 
contractor.  The  PCOs  included  in  the  twenty-five  percent 
had  found  it  necessary  to  go  back  to  the  contractor  numer¬ 
ous  times  to  extract  the  needed  information  one  bit  at  a 
time  for  both  prime  contractors  and  subcontractors .  One 
PCO,  in  desperation,  had  resorted  to  a  $  5-million  system 
definition  contract  in  an  attempt  to  tie  down  a  more  de¬ 
finite  plan  and  more  definite  information.  However,  this 
degree  of  difficulty  did  not  appear  to  be  widespread.  Once 


again,  PCOs  reiterated  that  the  contractors  wanted  the  con¬ 
tract  enough  to  cooperate. 


Question  #11.  Do  you  feel  that  your  source  selection  was 
affected  by  a  contractor's  reluctance  to  reveal  technological 
information  in  his  proposal? 

This  question  sought  the  PCOs*  opinions  concerning 
the  affect  of  contractor  reluctance  to  reveal  technological 
information.  In  consonance  with  the  previous  question, 
only  twenty-five  percent  felt  that  this  contractor  reluct¬ 
ance  had  affected  the  source  selection  process  in  any  way. 
These  PCOs  believed  that  the  length  of  time  required  for  the 
source  selection  process  had  been  increased  and  the  costs 
had  escalated.  However,  we  were  unable  to  uncover  any 
evidence  to  suggest  a  general  magnitude  of  such  time  and 
cost  expenses  for  the  entire  group  of  contracts .  The  rela¬ 
tive  magnitudes  are  discussed  in  the  data  analysis  section 
of  this  chapter. 

Question  #  12.  If  you  knew  that  a  contractor  withheld  tech¬ 
nical  information  from  his  technical  proposal  would  this 
lessen  his  chances  of  winning  the  contract? 

Next  a  more  general  question  was  asked  of  the  inter¬ 
viewees.  If  a  PCO  was  faced  with  a  situation  in  which  it 
was  known  that  a  contractor  had  withheld  technical  informa¬ 
tion,  would  this  lessen  the  contractor's  chances  of 
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winning  the  award?  The  responses  strengthened  our  original 
belief  that  a  problem  does  exist.  Fifty-five  percent  of  the 
PCOs  indicated  that  if  the  government  believed  that  the 
technical  information  was  significant  and  was  desired,  the 
contractor's  relative  position  would  be  in  jeopardy.  Ques¬ 
tion  #  8  reflected  sixty-five  percent  of  the  PCOs  having 
direct  experience  with  contractor  reluctance  to  divulge 
this  requested  technical  information.  Question  #  10  indi¬ 
cated  that  twenty-five  percent  of  the  PCOs  were  experiencing 
difficulty  in  technically  evaluating  contractor  proposals 
because  of  the  withholding  of  technical  information.  This 
places  the  contractor  in  a  position  of  being  forced  to  co¬ 
operate  by  sharing  technical  information  with  the  government 
in  order  to  protect  its  relative  competitive  position.  How¬ 
ever,  if  the  contractor  has  reason  to  believe  that  these 
data  will  not  be  adequately  protected  from  disclosure,  its 
relative  competitive  position  may  be  jeopardized  by  such  co¬ 
operation.  This  could  present  the  contractor  with  quite  a 
dilemma. 

An  unexpected  response  to  this  question  indicated 
that,  while  fifty-five  percent  of  the  PCOs  interviewed  be¬ 
lieved  that  the  withholding  of  technical  information  could 
lessen  a  contractor's  chances  for  award,  an  additional 
thirty  percent  believed  that  technical  information  is  not  an 
issue  in  many  cases  because  the  award  decision  is  primarily 
a  political  decision.  These  PCOs  stated  that  the  RFP  is 


issued  only  to  qualified  contractors.  Any  technical  ques¬ 
tions  or  difficulties  can  be  resolved  sifter  the  award  has 
been  made. 

Question  #  13.  Do  you  feel  that  you  can  offer  contractors 
the  level  of  incentives  and  protection  necessary  to  insure 
that  state-of-the-art  technology  is  included  in  their  tech¬ 
nical  proposals? 

Our  purpose  in  asking  this  question  was  to  determine 
the  government's  ability  to  offer  the  contractor  the  level 
of  incentives  and  protection  necessary  to  insure  that  state- 
of-the-art  technology  is  included  in  their  technical  pro¬ 
posals.  In  an  attempt  to  answer  .research  objective  1.0  and 
research  question  1.4,  the  PCOs  were  asked  to  give  us  their 
opinions.  Only  one  PCO  felt  that  he  was  unable  to  provide 
an  adequate  level  of  incentives/protection.  This  PCO  had 
experienced  reluctance  by  contractors  to  share  technical  in¬ 
formation  with  the  government  and  believed  that  an  adequate 
level  of  incentives/protection  would  serve  to  diminish  this 
reluctance.  Despite  this  PCO's  logic,  ninety-five  percent 
of  the  interviewees  felt  that  they  were  equipped  to  offer 
the  contractors  adequate  incentives/protection.  Upon  fur¬ 
ther  questioning,  only  four  of  these  PCOs  were  relying  upon 
the  standard  data  clauses.  The  remaining  15  PCOs  were  using 
an  array  of  tools  ranging  from  tailored  special  provisions 
to  complex  interface  or  integration  documents.  Several  of 
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them  indicated  that  they  had  created  whatever  clauses 
seemed  to  encourage  the  contractors  and  would  still  survive 
JAG  review.  They  admitted  that  JAG  had  approved  some  of  the 
clauses  because  they  were  totally  harmless  to  the  government 
and  totally  ineffective  for  the  contractors. 

Question  #  14.  Are  there  any  comments  you  would  like  to 
make  concerning  contractor  technical  proposals  and  the  with¬ 
holding  of  some  technical  information/capability  from  those 
proposals? 

This  question  allowed  the  PCOs  the  opportunity  to 
expound  on  any  opinions  or  comments  concerning  contractor 
technical  proposals  and  the  withholding  of  technical  informa 
tion  by  contractors.  Several  of  the  PCOs  felt  that  the  con¬ 
tractual  instrument  was  not  the  only  method  for  providing 
more  incentives/protection  to  the  contractor.  The  source 
selection  process  itself  was  considered  to  be  suspect  in  the 
search  for  reasons  for  contractor  reluctance  to  provide 
technical  data  to  the  government.  The  PCOs  believed  that 
the  number  of  individuals  involved  who  had  access  to  con¬ 
tractor  information  was  excessive.  One  PCO  even  developed 
an  elaborate  plan  for  subdividing  the  contractor's  informa¬ 
tion  and  assigning  each  subdivision  to  an  appropriate  team. 
In  this  way,  he  was  able  to  limit  the  number  of  individuals 
having  access  to  all  of  the  contractor's  technical  data  to 
three  people.  Another  PCO  stated  that  with  so  many 


individuals  involved,  it  would  be  impossible  to  eliminate 
technical  transfusion  entirely.  This,  he  believed,  stemmed 
from  a  basic  difference  in  philosophies.  The  engineer's 
main  concern  is  to  get  the  best  product  for  the  government, 
while  the  PCO's  main  objective  is  to  get  the  best  buy  for 
the  government  —  all  things  considered. 

The  other  factor  mentioned  that  has  an  effect  on  the 
contractor  and  its  concern  for  its  technical  data  is  the 
political  consideration.  Some  of  the  PCOs  felt  that  the  con¬ 
tractors  must  be  aware  that  technical  considerations  are  not 
always  the  only  key  capable  of  opening  the  door  to  an  award. 
Politics  also  plays  a  role.  This  must  have  some  effect  on 
the  contractor's  reluctance  to  divulge  certain  state-of-the- 
art  technology. 

Question  #  15.  Do  you  have  any  recommendations  in  regard  to 
specific  contractual  instruments  which  could  be  employed  or 
developed  to  insure  state-of-the-art  technology  is  not  with¬ 
held  from  technical  proposals? 

When  the  PCOs  were  asked  for  recommendations  for 
specific  contractual  instruments  which  could  be  used  to  in- 
centivize  and  protect  the  contractor,  each  suggested  his  or 
her  own  specialized  clause.  These  were  collected  and  studied 
to  aid  in  the  development  of  a  model  clause  to  be  used  in 
Phase  II  of  our  research. 
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Interview  Questions  -  Contractors 

The  interviews  with  contractors  consisted  of  8  basic 
questions  which  are  outlined  in  the  interview  guide  (Appen¬ 
dix  D) .  The  interviews  were  conducted  in  the  same  manner 
as  contracting  officer  interviews.  The  questions  were  asked 
in  the  order  shown  on  the  guide,  and  the  responses,  which 
were  of  a  subjective  nature,  were  summarized  by  question  to 
answer  specific  research  questions  associated  with  our  re¬ 
search  objectives. 

Question  #  1 .  How  many  proposals  (solicited  and  unsolicited) 
did  you  submit  to  the  Air  Force  during  FY  79? 

In  order  to  classify  the  contractors  interviewed, 
each  interviewee  was  first  asked  the  number  of  proposals  sub¬ 
mitted  to  the  Air  Force  in  FY  79-  In  every  case,  the  number 
exceeded  100,  enabling  us  to  document  the  respondents'  par¬ 
ticipation  in  government  contracting  and  acquisition.  The 
contractors  interviewed  were  also  listed  by  the  PCOs  as 
major  aerospace  contractors  with  which  they  frequently  ne¬ 
gotiated. 

Question  #  2.  Of  the  proposals  which  were  submitted,  did 
any  contain  what  you  perceived  to  be  state-of-the-art  tech¬ 
nology,  which  your  competitors  probably  did  not  possess? 

One  hundred  percent  of  the  contractors  interviewed 
stated  that  they  do  submit  source  selection  sensitive  data 
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to  the  government  in  their  proposals.  In  all  cases,  the  con¬ 
tractors  believe  that  some  of  this  information  definitely 
contains  technology  beyond  the  capabilities  of  their  com¬ 
petition. 

Question  #  3. a.  Has  the  possibility  that  anyone  may  obtain 
copies  of  your  technical  proposals  from  the  Air  Force  led 
you  to  refrain  from  submitting  a  proposal  on  a  procurement 
which  you  otherwise  would  have  bid  upon? 

Despite  the  earlier  contention  by  the  PCOs  that  the 
contractors  are  so  eager  for  business  that  they  would  not 
even  consider  refraining  from  submitting  a  proposal,  sixty 
percent  of  the  industry  respondents  stated  that  they  do 
refrain  when  they  believe  that  success  is  predicated  upon 
proper  timing.  An  example  would  be  a  two-phase  program  in 
which  the  second  phase  is  highly  desirable.  If  a  contractor 
believes  that  the  revelation  of  technical  data  from  the  first 
phase  may  enable  the  competition  to  compete  successfully  on 
the  second  phase,  the  contractor  may  elect  to  wait  for  the 
second  phase  while  continuing  to  develop  its  concept  with 
IR&D  (Independent  Research  &  Development)  money  and  techni¬ 
cal  information  being  developed  during  the  first  phase. 

One  contractor  indicated  that,  while  the  possibility 
of  release  of  technical  data  is  not  the  sole  reason  for  re¬ 
fraining  from  submission  of  a  proposal,  it  contributes  sig¬ 
nificantly  to  such  a  decision.  This  problem  is  not,  he 
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continued,  confined  to  the  data  of  prime  contractors.  The 
contractors  also  stated  that  the  technical  data  of  subcon¬ 
tractors  contribute  to  the  dilemma.  One  respondent  cited  a 
case  in  which  his  company's  inability  to  provide  detailed 
information  about  a  subcontractor's  process  caused  his  com¬ 
pany  to  lose  the  competition  even  though  he  believed  that 
the  information  was  not  required  by  the  government  in  order 
to  technically  evaluate  the  proposal. 

Question  #  3.b.  Has  the  possibility  of  release  of  your  tech¬ 
nical  proposals  led  you  to  alter  your  final  price  in  a  pro¬ 
curement? 

None  of  the  contractors  believed  that  the  possibility 
of  release  of  technical  data  has  had  an  effect  upon  the  final 
negotiated  price  on  any  of  their  contracts.  This  answer, 
however,  is  of  special  interest  when  compared  to  the  answers 
to  the  c.  portion  of  this  question. 

Question  #  3.c.  Do  you  perceive  that  the  possibility  of  re¬ 
lease  of  technical  proposals  has  led  your  competition  to 
alter  their  final  price  in  a  procurement? 

Thirty  percent  of  the  interviewees  believed  that 
their  competition  does  alter  price  in  an  attempt  to  recover 
a  portion  of  the  loss  incurred  by  government  release  of  tech¬ 
nical  data  to  competitors.  They  contended  that  this  practice 
is  more  prevalent,  in  their  opinion,  for  sole  source  and 


parallel  development  programs.  Without  the  threat  of  com¬ 
petition  and  the  possibility  of  losing  the  contract  due  to 
price,  the  respondents  believed  that  some  of  their  competi¬ 
tors  have  included  extra  dollars  in  order  to  compensate  for 
the  loss  of  valuable  technical  data. 

Question  #  3.d.  Has  the  possibility  of  release  of  your 
technical  proposals  substantially  affected  the  negotiation 
time  on  procurements  in  which  you  submitted  proposals? 

Thirty  percent  of  the  contractors  had  experienced 
protracted  negotiations  due  to  a  perception  of  insufficient 
protection  for  source  selection  sensitive  data.  These  con¬ 
tractors,  however,  estimated  that  ten  to  twenty  percent  of 
the  delays  in  negotiations  are  directly  attributable  to 
data  questions.  One  contractor  cited  an  example  in  which 
negotiations  took  so  long  that  the  funding  was  lost  in  a 
budget  cut  before  agreement  was  reached.  The  program  man¬ 
ager  was  so  angry  that  the  contractor  expressed  real  concern 
that  future  competitions  in  which  his  company  becomes  in¬ 
volved,  may  be  in  jeopardy.  He  believed  that  a  request  for 
use  of  the  clause  entitled  "Limited  Rights  in  Technical 
Data"  (DAR  7-104.9)  is  considered  by  government  personnel  to 
be  an  insult  to  their  integrity. 

The  Limited  Rights  clause  defines  limited  rights  as 

rights  to  use,  duplicate,  or  disclose  techni¬ 
cal  data  in  whole  or  in  part  by  or  for  the  Govern¬ 
ment,  with  the  express  limitation  that  such 
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technical  data  may  not  be  released  outside  the 
Government,  or  used,  duplicated,  or  disclosed,  in 
whole  or  in  part,  for  manufacture  or  procurement, 
except  for  .  .  .  and  .  .  .  released  to  a  foreign 
government  .  .  .  [jlOs  7*  104-9]. 

The  use  of  the  clause  is  proposed  by  the  contractor  in  its 

proposal  to  protect  data  developed  at  private  expense. 

Question  #  4.  Do  you  identify  source  selection  sensitive 
material  in  your  technical  proposals  through  restrictive 
legend  or  other  means? 

The  unanimous  response  to  this  question  can  be  sum¬ 
marized  in  the  words  of  one  of  the  interviewees,  "You’re 
damned  right  we  do!"  However,  even  though  the  use  of  a  re¬ 
strictive  legend  seems  relatively  straightforward,  problems 
arise  here  as  well. 

Inconsistencies  between  agencies  seem  to  be  wide¬ 
spread.  The  DAR  calls  for  the  use  of  a  specific  restrictive 
legend  on  each  proposal  page  containing  proprietary  data. 

This  legend  can  be  found  in  DAR  3-507  and  limits  the  govern¬ 
ment's  right  to  use  that  data  only  for  evaluation  purposes. 
All  of  the  contractors  believed,  due  to  discouraging  comments 
from  government  personnel,  that  a  proposal  including  several 
pages  marked  with  the  restrictive  legend  would  be  discrim¬ 
inated  against  during  the  technical  evaluation  phase  of  the 
acquisition.  In  order  to  avoid  this  demerit,  the  inter¬ 
viewees  indicated  several  procedures  for  handling  the  pro- 
blemt 
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(1)  Gamble  on  the  increased  chances  of  winning  a 
competition  by  not  marking  data  considered  to  be  proprie¬ 
tary  and/or  source  selection  sensitive; 

(2)  Include  the  restrictive  legend  for  proposal 
purposes  without  requesting  the  limited  rights  clause  in 
any  resultant  contract; 

(3)  Mark  the  source  selection  sensitive  data  "For 
Government  Use  Only"  as  a  compromise.  This  alerts  the  gov¬ 
ernment  to  the  sensitive  nature  of  the  data  without  formally 
restricting  their  use;  or 

(4)  Include  both  the  restrictive  legend  and  the 
limited  rights  clause  in  the  proposal  optimistic  that  they 
will  not  earn  a  demerit  during  the  evaluation  process. 

The  contractors  also  stated  that  some  PCOs,  as 
standard  operating  procedure,  include  a  clause  in  their  RFPs 
which  forbids  the  use  of  the  limited  rights  clause  in  any 
proposal.  The  use  of  such  a  clause  will  result  in  the  pro¬ 
posal  being  declared  non-responsive.  Needless  to  say,  the 
contractors  resent  this  restriction  by  the  PCOs. 

Forty  percent  of  the  respondents  mentioned  that  data 
rights  disagreements  often  stem  from  a  subcontractor’s  reluc¬ 
tance  to  allow  the  prime  to  share  technical  data  with  the 
government.  The  subcontractor's  requirement  to  include  the 
limited  rights  clause  can  place  the  prime  in  an  impossible 
position. 

The  responses  to  this  question  served  to  indicate 
the  level  of  incentives  and/or  protection  currently  offered 
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to  the  contractor  in  answer  to  research  question  1.4.  The 
remainder  of  the  questions  were  designed  to  investigate 
research  objective  2.0s 

Develop  a  decision  model  which  will  prescribe 
the  levels  of  incentives  and/or  protection  which 
the  government  must  utilize  to  assure  alternative 
design  concepts  throughout  the  acquisition  process. 

Question  #  5.  Do  you  know  of  instances  where  the  government 
has  released  technical  proposals,  or  portions  thereof,  that 
you  have  submitted  that  contain  what  you  consider  proprietary 
technical  information? 

Fifty  percent  of  the  interviewees  knew  of  specific 
cases  in  which  source  selection  sensitive  data  were  released 
to  competitors.  One  of  the  cases  was  under  the  auspices  of 
the  Freedom  of  Information  Act.  The  remainder  of  the  exam¬ 
ples  cited  were  instances  in  which  technical  transfusion  re¬ 
sulted  from  conscious  or  unconscious  actions  on  the  part  of 
government  personnel.  These  known  cases  coupled  with  rumors 
circulated  in  contracting  circles  appeared  to  be  cause  for 

great  concern  to  the  contractors  interviewed. 

\ 

Question  #  6.  Has  the  information  that  the  Air  Force  re¬ 
leased  enabled  another  company  to  compete  with  you  in  pro¬ 
curements  in  which  they  could  not  compete  prior  to  the  re¬ 
lease? 


The  same  contractors  that  responded  affirmatively 
to  question  #  5  believed  that  their  competitive  position  had 
been  damaged  by  the  government  release  of  source  selection 
sensitive  data.  This  concern  explains  the  reluctance  by 
sixty  percent  of  the  interviewees  to  submit  technical  pro¬ 
posals  to  the  government  unless  they  believed  that  the  timing 
was  right  for  a  profitable  award. 

Question  #  7.  Is  there  a  level  of  protection/incentives 
which  the  Government  must  offer  before  you  will  submit  a 
proposal  which  contains  source  selection  sensitive  data? 

All  of  the  contractors  qualified  their  answers  to 
this  question  by  indicating  that  some  data  are  considered  to 
be  more  sensitive  than  others.  The  relative  sensitivity  of 
the  data,  as  well  as  the  relative  importance  and  timing  of 
the  solicitation,  contributes  significantly  to  the  contrac¬ 
tor’s  decision  to  submit  a  proposal.  The  greater  the  re¬ 
wards  associated  with  winning  a  contract,  the  greater  are 
the  risks  the  contractors  are  willing  to  take . 

One  hundred  percent  of  the  interviewees,  however, 
suggested  preferred  government  methods  of  protecting  their 
source  selection  sensitive  data  in  answer  to  research  ques¬ 
tion  2.1.  These  methods  include: 

(1)  Allowing  the  usage  of  the  limited  rights  clause 
in  those  instances  involving  data  developed  at  private  ex¬ 


pense  ; 


(2)  Obtaining  contractual  government  agreement  not 
to  release  source  selection  sensitive  data  prior  to  formal 
source  selection; 

(3)  Clearly  stating  an  official  government  policy 
and  vigorously  enforcing  it; 

(4)  Restricting  reference  to  sensitive  data  for  a 
period  of  two  years; 

(5)  Handling  sensitive  contractor  data  in  the  same 
manner  that  contractors  are  required  to  handle  classified 
material  (use  of  hand  receipts,  return  of  all  copies  or 
destruction  certificates,  use  of  source  selection  sensitive 
cover  sheets); 

(6)  Notifying  contractors  openly  of  any  Air  Force 
intention  to  use  technical  transfusion  during  fact  finding; 
and 

(7)  Requiring  all  individuals  handling  the  source 
selection  sensitive  data  to  sign  written  statements  pledging 
secrecy. 

Question  #  8.  Do  you  have  any  recommendations  in  regard  to 
contractual  instruments  which  the  Government  should  employ 
or  develop  which  would  reduce  any  reluctance  you  may  have  to 
include  state-of-the-art  technology  in  your  technical  pro¬ 
posals? 


The  proper  utilization  of  the  limited  rights  clause 
was  discussed  by  all  of  the  contractors  interviewed  for 
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protection  of  data  developed  at  private  expense.  The  use 
of  a  contractual  clause  designed  to  protect  source  selection 
sensitive  data  prior  to  formal  source  selection  was  suggested 
hy  fifty  percent  of  the  respondents  for  data  considered  to 
he  competition-sensitive  hut  not  developed  at  private  ex¬ 
pense.  They  suggested  the  development  of  a  standard  clause 
within  the  limitations  of  the  DAR  in  response  to  research 
question  2.2. 

Data  Analysis 

As  discussed  in  Chapter  2,  our  purpose  in  obtaining 
the  number  of  competing  offers  and  the  costs  before  and  after 
negotiation  for  both  groups  of  contracts  was  to  determine  if 
relationships  exist  between  government  ownership  of  source 
selection  sensitive  data  and  cost  and  competition.  The  re¬ 
lationship  with  time  was  to  be  considered  in  this  analysis. 
However,  it  was  found  that  a  basis  for  comparison  did  not 
exist  since  all  but  one  of  the  PCOs  felt  that  their  negotia¬ 
tions  were  of  normal  lengths. 

The  existence  of  relationships  between  the  vari¬ 
ables  was  tested  through  the  use  of  the  T-test  described  in 
detail  in  Chapter  2.  In  performing  the  test,  we  used  the 
standard  SPSS  subprogram  T-test.  Therefore,  test  statistics 
were  not  computed  manually;  however,  a  listing  of  the  applic¬ 
able  formulae  for  the  tests  is  shown  in  Appendix  E.  In 
using  this  test,  we  established  a  data  file  of  the  informa¬ 
tion  acquired  during  the  PCO  interviews  and  included 
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directly  in  the  computer  program.  The  file  contained  the 
number  of  cases  (interviews)  and  consisted  of  three  vari¬ 
ables.  The  first  data  entry  in  each  case  was  the  number  of 
competing  offers.  The  second  variable  was  the  cost  of  nego¬ 
tiation  which  was  computed  as  the  difference  between  the 
contract  cost  estimated  before  negotiation  and  the  cost  after 
negotiation.  The  third  entry  signified  the  group  to  which 
the  contract  belonged;  that  is,  A  (1)  signified  contracts 
which  had  problems  with  source  selection  sensitive  data  while 
A  (2)  signified  contracts  in  which  data  were  not  identified 
as  a  problem.  For  example,  the  data  file  for  interview 
number  one  was  as  follows: 

1  3*1  1 

A  complete  listing  of  all  data  files,  the  programs  used  to 
run  the  T-tests,  and  the  program  output  are  contained  in 
Appendix  F. 

The  range  in  values  for  the  competition  variable  was 
a  low  of  (1)  to  a  high  of  (8)  in  the  group  of  contracts  in 
which  source  selection  sensitive  data  were  not  an  issue. 

The  group  which  had  data  problems  received  from  (1)  to  (4) 
offers.  As  discussed  earlier  in  this  chapter,  the  number  of 
contractors  qualified  to  perform  on  a  particular  contract  is 
believed  to  be  more  dependent  on  the  type  of  system  or  item 
the  Government  was  trying  to  buy  than  upon  any  other  factor. 
The  cost  figures  were  also  dependent  upon  the  system  involved 


56 


although  to  a  lesser  degree.  Contracts  which  involved  large 
production  runs  were  more  susceptible  to  decreases  in  cost 
following  negotiation  while  those  involving  a  smaller  number 
of  high  technology,  state-of-the-art,  production  items 
tended  to  increase  cost.  In  the  group  of  contracts  in  which 
source  selection  sensitive  data  were  an  issue  the  cost  fig¬ 
ures  ranged  from  a  low  of  (-1.5)  million  dollars  to  a  high 
of  (17)  million  dollars.  One  of  the  factors  which  increased 
costs  was  a  number  of  additional  people  required  to  support 
parallel  testing  and  evaluation  which  in  one  program  amounted 
to  (29)  additional  people  and  accounted  for  (3)  million  dol¬ 
lars  in  increased  costs.  Another  factor  which  increased 
costs  was  the  requirement  for  complicated  interface  docu¬ 
ments  which  delineated  contractors'  responsibilities  in  the 
sharing  of  data.  In  one  program,  for  example,  the  cost  of 
preparing  the  interface  document  alone  was  over  (60, 000) 
dollars . 

In  the  group  of  contracts  in  which  data  were  not 
considered  a  problem,  cost  savings  were  evidenced  in  (7)  out 
of  (10)  contracts.  As  seen  in  the  data,  the  values  for  cost 
ranged  from  (-98)  million  dollars  to  (+.8)  million  dollars. 
The  98  million  dollar  figure  was  the  savings  resulting  from 
negotiation  of  a  large  production  contract  and  was  rather 
unique  due  to  its  magnitude.  While  cost  savings  were  real¬ 
ized  in  seven  contracts  in  this  group,  as  compared  to  only 
one  in  the  group  which  experienced  data  problems,  the  task 
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was  to  determine  if  the  difference  was  significant  at  the 
(90)  percent  level  of  confidence.  The  (90)  percent  level 
was  chosen  by  the  researchers  as  an  appropriate  level  of 
risk  for  this  research  situation  due  to  the  subjective  nature 
of  the  questions  and  the  information  we  were  attempting  to 
capture . 

The  hypotheses  used  in  both  tests  as  stated  in 
Chapter  2  were: 

Hq:  ~  °!  no  relationship  exists 

ILj  42_!I1  ^  °5  relationship  exists. 

In  order  to  use  the  T-test  procedure  some  important  assump¬ 
tions  must  be  made  and  were  made  in  this  research  effort. 

The  first  was  that  the  two  populations  from  which  the  samples 
were  drawn  were  normal  or  the  departures  were  not  too  marked. 
The  second  assumption  that  was  made  was  that  the  two  popula¬ 
tions  had  the  same  variance.  Finally,  independent  random 
samples  must  be  drawn  from  the  two  populations.  As  discussed 
in  Chapter  2,  we  recognize  that  a  purposive  non-random  sample 
limits  the  confidence  with  which  we  can  make  inferences  to 
the  populations  of  concern.  Yet  the  samples  were  chosen 
with  the  intent  to  provide  a  reasonable  cross-section  of  con¬ 
tract  situations  and  will  be  treated  as  independent  random 
samples  for  the  purposes  of  these  tests. 

The  first  test  performed  was  to  test  if  a  relation¬ 
ship  existed  between  source  selection  sensitive  data  and  the 
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number  of  offers  received  for  the  contract  situations  of 
concern.  The  a  value  used  was  (.10)  since  we  are  looking  at 
a  (90)  percent  confidence  level.  The  test  results  are  shown 
on  page  2  of  Appendix  F.  Since  we  have  assumed  equal  var¬ 
iances,  the  pooled  variance  estimate  was  used  in  the  inter¬ 
pretation  of  results.  As  seen  in  the  results,  there  was 
little  difference  in  the  means  of  the  two  samples.  The 
group  of  contracts  that  had  no  data  problems  had  a  mean  of 
(3*0)  while  the  group  with  data  problems  had  (2.4)  as  the 
mean  number  of  offers. 

The  first  portion  of  the  test  for  significance  was 
to  develop  a  confidence  interval  around  the  suspected  dif¬ 
ference  between  the  means.  The  formulas  used  were* 

L  <  u2-p^  <  u 

where  L  =  D  -  t(l-a/2;  n^  +n2~2)  S(5) 

U  =  C  +  t( 1 -a/2;  nx  +n2-2)  S(D) 

The  two-tailed  test  was  appropriate  as  called  for  in  the 
hypotheses.  For  an  a/2  of  (.05)  and  18  degrees  of  freedom 
the  critical  value  of  t  as  listed  in  a  statistical  table  of 
values  was  (1.73*0  •  The  resulting  confidence  interval  was 
-.9337  <  h2~lAl  <  2 •  H37  which  encompasses  the  value  of  #. 

As  a  result  we  could  not  reject  the  null  hypothesis.  We 
performed  three  more  tests  to  insure  that  we  had  made  the 
proper  conclusion. 

The  second  test  that  we  made  was  a  check  of  the 
probability  values.  The  a  for  the  test  was  (.10)  and  was 
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compared  to  the  two-tailed  probability  taken  from  the  computer 
output  in  Appendix  F.  Since  the  two-tailed  probability  in 
the  test  results  was  (.50 6),  which  was  greater  than  the  a  of 
(.10),  we  once  again  concluded  HQ. 

The  third  check  was  a  comparison  of  the  t  value  con¬ 
tained  in  the  test  results  with  a  set  of  action  limits.  By 
referring  to  a  statistical  table  of  critical  values  for  the 
t  distribution,  at  an  a/2  of  (.05)  and  (18)  degrees  of  free¬ 
dom,  the  critical  value  of  t  was  determined  to  be  (1.73*0* 
Thus,  for  a  two-tailed  test  we  could  reject  the  null  only 
if  the  test  value  fell  outside  the  limits  -1.73**  to  1.73**. 

As  seen  in  the  results,  the  value  of  the  test  statistic 
t*  is  (.68)  which  is  well  within  the  limits.  Figure  1  illus¬ 
trates  where  the  t*  value  falls  in  the  distribution  and  shows 
the  action  limits  which  have  the  t  values: 

At  =  -1.73**  and  A2  =  +1.73**. 

The  final  step  was  to  check  the  results  with  a  de¬ 
cision  rule.  The  construction  of  the  decision  rule  entailed 
the  placement  of  the  action  limits  as  shown  in  Figure  1, 

Thus  the  decision  rule  used  was: 

If  -1.53**  <  0  <  1.53**  conclude  HQ 

If  -1.53**  >  D  or  D  >  1.53**  conclude 

The  calculated  value  of  the  difference  of  means  is  (.6)  which, 
according  to  our  decision  rule,  is  within  the  action  limits. 
Thus  we  must  conclude  HQ,  since  there  is  insufficient 
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t*  =  .68 
t 


Aj  =  0  -t ( l -a/ 1 }  n1  +  n2-2)  S(S) 

Aj  =  -t(. 95; 18) (.8845) 

Aj_  =  -1.534 

A2  =  0  +  t  ( 1 -a/ 1 ;  nx  +  n2-2)  S(D) 

Ag  =  +  t(. 95s 18)(. 8845) 

A2  =  1.534 

Figure  1.  Competition  Test 

evidence  to  suggest  that  at  the  (.10)  level  of  significance 
a  relationship  exists  between  source  selection  sensitive  data 
and  the  level  of  competition  of  the  populations  of  concern. 

The  second  relationship  which  we  tested  was  to  de¬ 
termine  if  a  statistically  significant  difference  existed 
between  the  costs  of  negotiation  of  the  two  groups  of  con¬ 
tracts.  The  test  results  for  cost  are  displayed  on  page  3 
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of  Appendix  F.  The  same  tests  were  performed  during  the 
interpretation  of  the  cost  results  as  were  used  in  the  level 
of  competition  analysis.  The  difference  in  means  between 
the  two  groups  was  calculated  to  be  (-18.572)  million  dol¬ 
lars. 

The  development  of  a  90  percent  confidence  around 
the  expected  difference  between  the  two  means  was  the  first 
step  in  the  test  for  significance.  Again,  a  two-tailed  test 
was  appropriate.  Thus,  the  formulas  previously  used  for  cal¬ 
culating  the  confidence  interval  for  the  number  of  offers 
were  used  in  this  case.  From  a  statistical  table  of  values 
for  the  t  distribution  we  obtained  for  an  a/2  of  (.05)  and 
(18)  degrees  of  freedom  a  critical  value  of  (1.73*0.  The 
resulting  confidence  interval  with  a  mean  difference  of  ■ 
(-18.572)  was  -35-698  <  -l.W-5-  Since  the  90  per¬ 

cent  confidence  interval  does  not  encompass  the  value  of  / 
we  rejected  the  null  hypothesis  (HQ) .  However,  three  more 
steps  were  taken  to  check  our  conclusion. 

A  test  of  probability  values  was  the  second  step  of 
our  test  of  significance.  The  a  for  this  test  was  (.10) 
which  was  compared  to  the  two-tailed  probability  contained 
in  the  pooled  variance  estimate  portion  of  the  results  on 
page  3  of  Appendix  F.  The  value  calculated  was  (.076). 

Since  the  test  value  of  (.076)  was  less  than  the  a  of  (.10) 
we  concluded  Hj  again. 

The  third  check  involved  comparison  of  the  value  of 


the  test  statistic  with  a  set  of  action  limits.  The  critical 
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value  of  t  for  a/2  of  (.05)  and  (18)  degrees  of  freedom  was 
(1-73*0  •  We  could  reject  the  null  hypothesis  only  if  the 
value  of  the  test  statistic  t*  fell  outside  the  critical 
action  limits  of  =  -1 . 73*+  and  Ag  =  +1.73*0  The  value  of 
t*  as  shown  in  the  results  was  (-1.88).  Thus  we  can  reject 
the  null  since  the  test  statistic  was  found  to  be  less  than 
action  limit  .  Figure  2  depicts  where  t*  fell  within  the 
D  distribution. 

The  final  step  was  the  construction  of  the  decision 
rule.  Figure  2  shows  the  calculations  of  the  action  limits. 


A1  .=  0  -t ( 1  -a/ 2 ;  nj  +  n2-2)  S(D) 
Ax  =  -t( .95;  18) (9 • 877 ) 

At  =  -17.127 

Ag  =  ff  +  t(l-a/2;  nj  +  n2-2)  S(D) 
Aj  =  +  t(.95»  18) (9 • 877) 

A1  =  +17.127 

Figure  2.  Cost  Test 
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Thus  the  decision  rule  used  wast 


If  -17.127  <  D  <  17-127  conclude  Hq 

If  -17.127  >  B  or  D  >  17.127  conclude 

The  calculated  difference  in  means,  as  stated  previously, 
was  (-18.572).  Thus  we  again  can  reject  Hq  and  conclude 
since  D  <  -17.127.  The  test  results  have  indicated  that  we 
can  he  90  percent  confident  that  a  relationship  exists  be¬ 
tween  the  presence  of  source  selection  sensitive  data  pro¬ 
blems  during  contract  negotiations  and  increased  costs 
for  those  negotiations.  For  the  contracts  which  we  studied 
there  was  a  mean  difference  of  over  18  million  dollars  be¬ 
tween  those  contracts  which  experienced  source  selection 
sensitive  data  problems  and  those  that  had  no  such  problems. 
We  found  that  the  mean  number  of  offers  received  in  contract 
situations  in  which  source  selection  sensitive  data  were 
not  a  problem  was  higher  than  contract  situations  which  had 
data  problems.  However,  we  found  that,  at  the  90  percent 
level  of  confidence,  the  difference  was  not  statistically 
significant.  Thus  we  could  not  determine  that  a  relation¬ 
ship  existed. 

Model  Development 

As  discussed  in  Chapter  2,  the  data  acquired  and 
analyzed  as  part  of  the  Phase  I  effort  were  considered  in 
the  construction  of  a  decision  model.  The  decision  model, 
in  the  form  of  a  clause  designed  to  protect  contractors' 


source  selection  sensitive  data  prior  to  formal  source 
selection  was  developed  consistent  with  a  concensus  of  the 
recommendations  obtained  during  the  interviews.  The  con¬ 
tractors'  definitions  of  "adequate”  incentives  and  the  limits 
of  incentives  and  protection  which  the  government  can  offer 
to  contractors  assisted  in  the  formulation  of  a  contractual 
clause . 

The  decision  model  developed  for  Fhase  II  of  the  re¬ 
search  effort  is  embodied  in  the  following  clauses 

In  the  event  an  exchange  of  information/data 
during  the  conduct  of  this  contract  results  in  access 
to  source  selection  sensitive  data,  the  Government 
hereby  agrees  not  to  utilize  such  data  to  acquire 
from  others  improved  designs  or  components  prior  to 
formal  source  selection. 

Test  of  the  Model 

To  test  the  decision  model,  the  ten  contractors  were 
asked  to  review  the  clause  and  provide  comments  on  its  per¬ 
ceived  effect  upon  past  negotiations  in  which  data  problems 
had  surfaced.  The  contractors'  subjective  comments  were 
then  analyzed  to  determine  the  perceived  impact  of  the  clause 
on  the  level  of  competition  and  the  time/cost  of  negotia¬ 
tions.  The  results  of  the  Hiase  II  data  collection  and 
analysis  are  presented  below. 

Decision  Model  Test  Results 

As  discussed  earlier,  the  model  clause  was  developed 
incorporating  the  suggestions  of  eighty  percent  of  the  con¬ 
tractors  interviewed. 
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In  the  event  an  exchange  of  information/data 
during  the  conduct  of  this  contract  results  in 
access  to  source  selection  sensitive  data,  the 
Government  hereby  agrees  not  to  utilize  such  data 
to  acquire  from  others  improved  designs  or  com¬ 
ponents  prior  to  formal  source  selection. 

The  model  was  then  tested  by  asking  the  industry 
representatives  to  consider  the  perceived  impact  of  the 
clause  upon  the  level  of  competition  and  the  time  and  cost  of 
the  negotiation  process.  Two  basic  questions,  research  ques¬ 
tions  3.1  and  3*2,  were  asked. 

Research  Question  3.1.  Would  the  use  of  the  incentives  and/ 
or  protection  required  by  the  decision  model  have  resulted 
in  greater  assurance  of  alternative  design  concepts  through 
increased  competition? 

There  was  general  agreement  among  the  respondents 
that  the  use  of  the  model  would  not  have  any  significant 
impact  on  a  contractor’s  decision  to  submit  a  proposal  in 
response  to  an  RFP.  If  the  model  clause  was  contained  in 
the  RFP,  the  decision  to  submit  would  still  be  dependent  on 
the  relative  timing  and  value  of  the  proposed  program  effort. 
At  the  proposal  stage,  the  contractors  are  more  concerned 
about  the  engineering  personnel  involved  in  the  technical 
evaluation  than  about  the  PCO.  While  none  of  the  inter¬ 
viewees  objected  to  the  inclusion  of  the  clause,  none  per¬ 
ceived  any  real  benefit.  Sixty  percent  mentioned  the  per¬ 
ception  that  the  engineering  goal  is  to  pursue  the  develop¬ 
ment  of  the  best  possible  product  often  to  the  point  of, 
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consciously  or  unconsciously,  participating  in  technical 
transfusion.  One  contractor  even  stated  a  belief  that  con¬ 
scious  experiments  have  been  conducted  to  assess  the  advan¬ 
tages  and  disadvantages  of  technical  transfusion.  The 
general  concensus  of  opinion  was  that  little  would  be  gained 
by  the  use  of  the  model  at  the  RFP  stage  because  few  tech¬ 
nical  personnel  read  the  RFP  that  closely. 

Research  Question  3.2.  Would  the  use  of  the  model  have  de¬ 
creased  the  time  and  cost  of  the  negotiation  process? 

Sixty  percent  of  the  contractors  perceived  that  use 
of  the  model  during  negotiations  would  have  decreased  the 
length  of  the  negotiation  process.  The  introduction  of  the 
clause  would  have  been  considered  an  act  of  good  faith  on 
the  part  of  the  government.  Although  its  real  value  and 
enforceability  were  considered  suspect,  to  question  its  con¬ 
tribution  would  be  to  question  the  integrity  of  the  govern¬ 
ment  personnel  involved  in  the  decision-making  process  as 
well  as  the  technical  effort  itself.  In  a  competitive 
atmosphere,  this  could  result  in  a  demerit. 

Another  benefit  associated  with  the  use  of  the  model 
is  the  introduction  of  the  government  policy  into  the  negotia¬ 
tion  process  while  all  concerned  personnel,  including  the 
technical  staff,  are  present.  The  contractors,  in  general, 
had  very  little  complaint  about  the  official  government 
policy  regarding  source  selection  sensitive  data,  e.g., 
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the  confidential  treatment  of  data  submitted  to  the  govern¬ 
ment  for  evaluation.  However,  they  felt  that  the  proper 
implementation  of  the  policy  is  neglected  at  lower  levels 
in  varying  degrees. 

Although  the  general  consensus  of  contractor  opinions 
did  not  reflect  a  belief  that  the  final  negotiated  contract 
price  was  affected  by  the  submission  of  source  selection 
sensitive  data  in  our  initial  Phase  I  interviews,  our  research 
findings  indicated  that  a  relationship  does  exist.  The  in¬ 
terviewees  were  questioned  again  in  Phase  II  to  determine  if 
the  use  of  the  model  clause  would  result  in  the  decreased 
cost  of  the  negotiation  process.  There  was  general  agree¬ 
ment  among  the  contractors  that  a  decrease  in  the  length  of 
negotiations  would  result  in  a  decrease  in  the  cost  as  well. 
The  rationale  behind  this  perception  is  the  belief  that,  if 
the  contractor  becomes  involved  in  protracted  negotiations 
as  the  result  of  a  disagreement  about  source  selection  sensi¬ 
tive  data,  the  contractor  may  not  be  as  agreeable  about  price 
negotiations.  Another  general  conclusion  of  the  interviewees 
is  that  time  is.  money  in  a  major  weapons  system  acquisition. 
Many  high  level  individuals  are  tied  up;  travel  expenses  are 
costly;  and  the  acquisition  of  necessary  equipment  is  delayed. 
Also,  the  result  of  a  lengthy  negotiation  may  be  a  more 
costly  solution  designed  to  protect  the  data.  These  views 
were  expressed  by  only  thirty  percent  of  the  contractors, 
however . 


Chapter  4 


FINDINGS  AND  RECOMMENDATIONS 

This  chapter  discusses  our  overall  findings  and  how 
they  related  to  the  research  objectives  and  questions,  our 
recommendations  for  a  policy  designed  to  protect  source  selec¬ 
tion  sensitive  data,  other  related  observations,  and  our  re¬ 
commendation  for  further  study. 

Achievement  of  Objectives/ 

Questions' 

The  results  of  this  research  effort  only  partially 
met  our  objectives.  The  first  objective  was  to  determine 
the  relationships  between  the  government  ownership  of  source 
selection  sensitive  data,  the  level  of  competition,  the  time/ 
cost  of  the  negotiation  process,  and  the  level  of  incentives. 
This  study  revealed  that*. 

(1)  a  relationship  between  sensitive  data  and  a  de¬ 
creased  level  of  competition  does  appear  to  exist;  however, 
for  the  purposes  of  this  research  effort,  the  relationship 
was  determined  to  be  insignificant  due  to  our  decision  to 
statistically  test  the  data  at  the  90#  confidence  level. 

This  decision  does  not  prove  that  no  relationship  exists. 

(2)  a  relationship  cannot  be  measured  between  gov¬ 
ernment  ownership  of  sensitive  data  and  the  length  of  time 
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required  for  the  negotiation  process  without  the  implementa¬ 
tion  of  procedures  designed  to  capture  more  detailed  in¬ 
formation  about  some  key  dates  during  negotiations; 

(3)  a  definite  direct  relationship  does  exist  be¬ 
tween  the  acquisition  of  source  selection  sensitive  data  and 
increased  costs  associated  with  the  negotiation  process;  and 

(4)  an  increased  level  of  incentives/protection  does 
increase  government  acquisition  of  source  selection  sensi¬ 
tive  data  by  decreasing  the  contractor's  reluctance  to  share 
such  data  with  the  government. 

This  first  objective  was  met  with  the  exception  of 
the  measurement  of  time  required  for  negotiation.  The  dif¬ 
ference  of  opinions  between  the  PCOs  and  the  contractors  in 
this  area,  coupled  with  the  contractor  perception  that  the 
use  of  the  decision  model  would  result  in  a  decreased  time 
for  negotiations,  led  to  our  belief  that  this  area  warrants 
further  study.  More  detailed  information  about  the  time  in¬ 
volved  in  the  negotiation  process  could  be  captured  on  future 
acquisitions  in  order  to  determine  the  validity  of  the  PCO 
and  contractor  perceptions. 

The  second  objective  was  to  develop  a  decision  model 
which  would  prescribe  a  level  of  incentives  and/or  protection 
which  the  government  must  utilize  to  assure  more  willingness 
on  the  part  of  the  contractor  to  share  alternative  design 
concepts  throughout  the  acquisition  process.  Based  upon  the 
responses  of  the  interviewees  during  Phase  I,  we  did  develop 
a  model  in  the  form  of  a  clause  used  during  Phase  II  of  the 


research. 
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The  third  objective  was  to  test  the  model  to  de¬ 


termine  if  its  use  resulted  in  a  perceived  increase  in  the 
level  of  competition  and  decrease  in  the  time/cost  of  negotia 
tions.  During  Phase  II  of  the  research,  we  did  test  the 
model.  Although  a  majority  opinion  was  obtained,  the  re¬ 
sults  were  evaluated  against  the  background  of  the  subjective 
nature  of  the  responses.  Based  upoi.  this  analysis,  we  be¬ 
lieve  that  the  use  of  our  model  would  result  in  a  decrease 
in  the  time  of  negotiation.  This,  in  turn,  could  result  in 
a  decrease  in  the  cost  of  negotiation;  however,  this  area 
warrants  further  study.  Our  research  indicates  that  a  de¬ 
crease  in  the  length  of  the  negotiation  process  may  result 
in  a  decrease  in  cost.  More  data  should  be  collected  to  de¬ 
termine  whether  such  a  relationship  does  exist.  However, 
first  it  would  be  necessary  to  capture  more  detailed  data 
on  the  times  associated  with  negotiations. 

Findings 

Our  research  was  approached  with  the  intent  of  col¬ 
lecting  enough  data  to  support  or  refute  our  a  priori  ex¬ 
pectations.  Our  findings  tended  toward  support  of  some  ex¬ 
pectations  and  refutation  of  others.  A  summary  of  these 
findings  includes; 

(1)  AFSC  PCOs  and  contractors  appear  to  have  sig¬ 
nificant  differences  of  opinion  regarding  the  impact  of  gov¬ 
ernment  ownership  of  source  selection  sensitive  data  on 
competition  and  negotiations. 


(2)  Government  ownership  of  source  selection  sensi¬ 
tive  data  does  significantly  impact  contractor  decisions 
throughout  the  entire  acquisition  process. 

(3)  The  level  of  incentives/protection  offered  by 
the  government  does  affect  the  contractor's  willingness  to 
share  sensitive  data  with  the  government. 

Recommendations  for  Management 
Action 

In  addition  to  our  research  findings,  several  obser¬ 
vations  were  derived  from  the  comments  of  the  interviewees: 

(1)  AFSC  should  critically  examine  its  policy  re¬ 
garding  sensitive  data.  Some  PCOs  are  forbidding  the  use  of 
the  limited  rights  clause  in  contractor  proposals  with  a 
penalty  of  a  declaration  of  nonresponsiveness  for  any  viola¬ 
tion. 

(2)  Earlier  participation  of  the  contractors  at 
the  Mission  Element  Needs  Statement  (MENS)  stage  should  re¬ 
sult  in  increased  data-sharing  by  the  contractors. 

(3)  Proper  use  of  draft  RFPs  increases  the  visi¬ 
bility  of  potential  data  problems  prior  to  the  solicitation, 
creating  the  opportunity  to  solve  any  disagreements  before 
becoming  involved  in  a  lengthy,  and  costly  negotiation. 

(4)  Source  selection  procedures  should  strive  for 
the  highest  level  of  secrecy  possible,  limiting  the  indivi¬ 
duals  involved  to  as  few  as  absolutely  necessary. 
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(5)  PCOs  should  adhere  to  the  "private  expense" 
test  for  determination  of  limited  rights  calling  upon  the 
Administrative  Contracting  Officer  (ACO)  and  auditor  to 
verify  that  funds  were  actually  expended  for  development  of 
the  data  in  question. 

Significance  of  Findings 

Test  results  of  the  model  clause  confirmed  our  a 
priori  expectations.  As  mentioned  earlier,  of  the  contractors 
interviewed,  sixty  percent  were  aware  of  specific  instances 
of  data  problems  which  led  to  decreased  competition,  in¬ 
creased  time,  or  increased  cost  associated  with  negotiations. 
After  testing  the  model,  six  of  the  contractors  perceived 
that  use  of  the  clause  would  have  had  a  favorable  impact  on 
the  resolution  of  the  contractor's  specific  data  problems. 
Although  the  results  were  not  dramatic,  our  research  in¬ 
dicates  that  a  level  of  incentives  and  protection  does  exist 
which  is  capable  of  impacting  the  contractor's  decision  to 
share  source  selection  sensitive  data  with  the  government. 

Recommendations  for  Further 
Study 

During  our  research  effort,  our  model  was  tested  on 
a  minimum  number  of  contractors  by  using  their  subjective 
judgment.  As  a  result  of  this  preliminary  research,  it 
should  now  be  possible  to  measure  PCO  and  contractor  opin¬ 
ions  less  subjectively.  This,  in  turn,  would  open  the  door 
to  additional  statistical  treatments  which  could  provide 
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insight  into  this  system,  e.g.,  the  inferred  existence  and 
nature  of  additional  relationships.  The  decision  model 
could  he  refined  by  developing  alternative  clauses.  These 
clauses  could  then  be  ranked.  This  ranking  process  could  be 
accomplished  in  the  following  manner: 

(1)  Several  clauses  could  be  developed  using  the 
suggestions  and  observations  of  this  research  effort.  These 
clauses  would  provide  a  basis  for  the  recommendations  of 
future  interviewees. 

(2)  A  7-  or  9-  point  Likert  scale  could  be  developed 
to  weigh  each  clause  against  the  other  clauses  or  against 
its  contribution  to  the  level  of  competition  and  the  time/ 
cost  of  the  negotiation  process. 

(3)  A  questionnaire  could  be  developed  in  order  to 
provide  a  larger  sample  comprised  of  PCOs,  contractors,  and 
attorneys . 

In  addition,  a  procedure  could  be  developed  to  cap¬ 
ture  more  detailed  information  regarding  the  length  of  the 
actual  negotiation  process  and/or  the  time  consumed  on  data 
problems  per  se. 

Related  Findings 

Another  area  related  to  source  selection  sensitive 
data  is  the  area  of  limited  rights.  AFSC  applies  the  official 
government  policy  regarding  limited  rights  inconsistently. 

This  seems  to  be  partially  due  to  a  difference  in 
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interpretation  and  partially  due  to  a  combined  zeal  for  data 
and  search  for  an  easy  way  out  of  data  disagreements.  This 
area,  too,  warrants  further  study. 
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DEPARTMENT  OF  THE  AIR  FORCE 
HEADQUARTERS  AIR  FORCE  LOGISTICS  COMMAND 
WRIGHT-PATTERSON  AIR  FORCE  BASE  OHIO  43433 


™™JANO  24M/>V107f? 

•  UB4CCT:  Suggested  Research  Topic 


TO,  AFIT/LSGM  (Lt  Col  Larry  Smith) 

1.  This  is  in  response  to  your  request  for  research  topics 
in  the  May  issue  of  the  NCMA  (Dayton  Chapter)  Newsletter. 

It  is  suggested  that  you  examine  the  appropriate  strategies, 
tactics  and  procedures  for  handling  data  that  is  generated 
under  contracts  during  long-term  parallel  development 
programs  where  the  data  is  sensitive  with  respect  to  those 
competitions . 

2.  For  example,  the  Air  Force  is  currently  conducting 
the  Design  Definition  Phase  of  the  Cruise  Missile 
Carrier  Aircraft  Program  (CMCA) .  There  are  three 
contractors  participating  in  this  phase.  All  three  of 
these  contractors  are  manufacturers  of  wide  bodied 
commercial  aircraft,  Boeing,  Lockheed  and  McDonnell 
Douglas . 

There  are  two  concerns  with  data.  The  Air  Force 
would  like  to  use  all  the  information  provided  under 
these  contracts  to  produce  the  RFP's  for  the  following 
contracts  and  in  the  source  selections  resulting  from 
those  RFP's.  Much  of  this  data  is  data  initially 
generated  for  their  commercial  aircraft  programs. 

Since  the  contractor's  wish  to  protect  their  perceived 
competitive  edge  in  the  commercial  programs  as  well  as 
the  CMCA  program  they  are  reluctant  to  grant  the  Air 
Force  substantial  rights  in  this  data. 

Compounding  this  problem  is  the  existence  of  the  Air 
Launched  Cruise  Missile  (ALCM)  Competitive  fly-off 
Program.  Boeing  and  General  Dynamics  (GD)  are  building 
missiles  for  that  program  and,  McDonnell  Douglas  is 
providing  the  navigation/guidance  package.  GD  and 
McDonnell  Douglas  are  reluctant  to  provide  missile  and 
guidance  information  to  Boeing  even  though  Boeing  needs 
that  information  in  order  to  modify  the  B-52  launch 
aircraft. 

All  of  the  contractors  on  the  ALCM  fly-off  Program 
are  reluctant  to  release  data  to  the  CMCA  contractors. 
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AT£C  -  £ifeline  of  the  Aerospace  Oeam 


This  is  one  small  look  at  a  set  of  interlocking, 
competitive  programs  which  stretch  over  a  multi  year 
period.  As  OMB  Circular  A-109  receives  more  attention 
this  problem  will  become  more  acute.  It  is  vital 
that  the  Air  Force  identify  the  appropriate  procedures 
strategies  and  tactics  to  deal  effectively  with  these 
problems  as  soon  as  possible. 

3.  If  you  desire  additional  information  with  respect  to 
this  recommendation  please  feel  free  to  contact  Captain 
C.  Brandon  Gresham  Jr.,,  extension  55270. 

FOR  THE  COMMANDER 


^/JOHN  A.  CIUCCI,  Lt  Col,  USAF 

Chief,  Procurement  Opinion  Division 
Directorate  of  Procurement  Law 
Office  of  the  Staff  Judge  Advocate 


APPENDIX  B 


LETTER  TO  INTERVIEWEES 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  FORCE  BUSINESS  RESEARCH  MANAGEMENT  CENTER  I  ATC ) 
WRIGHT-PATTERSON  AIR  FORCE  BASE.  OHIO  45433 


You  have  been  selected,  as  a  contracting  officer  (industry  representative) 
to  participate  In  an  Air  Force  Institute  of  Technology  (AFIT)  research 
thesis  on  the  relationships  of  source  selection  sensitive  data  to  the  level 
of  competition  and  the  time/cost  of  negotiation  efforts  in  a  major  systems 
contract  process.  The  objective  of  this  research  thesis  is  to  develop  a 
decision  model  which  will  predict  the  levels  of  incentives  and/or  protection 
which  the  government  must  utilize  to  insure  increased  levels  of  competition 
and  decreased  time/cost  of  negotiations. 

Within  the  next  three  weeks,  Ms.  Linda  Allen  or  Captain  Michael  J.  Hubert 
will  contact  you  to  arrange  for  a  convenient  time  for  a  personal  interview 
with  you  on  this  subject.  The  interview  will  last  approximately  one  hour 
and  will  follow  the  attached  interview  guide.  Since  the  validity  of  the 
research  results  will  depend,  in  part,  on  the  independence  of  the  responses, 
please  do  not  discuss  the  interview  questions  with  others  in  your  office. 

Your  responses  will  be  kept  confidential.  All  responses  will  be  reported 
by  categories  only  so  that  no  specific  response  will  be  attributed  to  any 
individual . 

This  research  effort  is  in  response  to  research  needs  of  the  Air  Force 
Business  Research  Management  Center  and  the  AFLC  Judge  Advocate  General's 
office  both  at  Wright-Patterson  AFB,  Ohio.  Questions  regarding  the  interview 
should  be  directed  to:  Captain  Michael  J.  Hubert  or  Ms.  Linda  Allen,  AFIT/ 
LS,  Area  Code  513-255-6569  or  513-879-1167. 

Your  cooperation  and  support  of  this  effort  would  be  greatly  appreciated. 


Executive  Director 


1  Atch 

Interview  Guide 
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AD-A087  220 
UNCLASSIFIED 


air  FORCE  INST  OF  TECH  KRIGHT-PATTERSON  AFB  OH  SCHOOL— ETC  F/G  15/5 
THE  RELATIONSHIP  OF  SOURCE  SELECTION  SENSITIVE  DATA  TO  COMPETIT— ETC (U> 
JUN  SO  L  K  ALLEN »  M  J  HUBERT 

AFIT-LSSR-3-B0  NL 


1.  How  many  offers  did  you  solicit  on  this  procurement 
and  to  which  contractors  were  RFPs  issued? 


2.  How  many  offers  did  you  receive? 


3. 


How  many  days  elapsed  between  receipt  of  purchase  re¬ 
quest  to  contract  award  in  this  procurement? 


4.  What  is  the  average  length  of  negotiation  time  for 
similar  contracts? 


5.  What  was  the  engineering  estimate  of  the  contract  cost 
contained  in  the  purchase  request? 


6.  What  was  the  actual  negotiated  contract  cost? 


7.  Of  the  technical  proposals  submitted,  were  any  identi¬ 
fied  as  containing  source  selection  sensitive  material? 

Yes _  No _ 

If  yes,  can  you  give  some  examples? 
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8.  Have  you  detected  a  reluctance  on  the  part  of  con¬ 
tractors  to  submit  technical  proposals  containing  their 
state-of-the-art  technology? 

Yes _  No _ 

If  yes,  how  did  you  detect  this  reluctance? 


9.  Have  any  contractors  discussed  with  you  the  problem  of 
disclosure  of  their  technical  proposals  which  could  con¬ 
tain  state-of-the-art  technology? 

Yes _  No _ 

If  yes,  what  were  their  ideas? 


10.  Have  you  experienced  difficulty  evaluating  technical 
proposals  in  which  the  contractor  has  apparently  with¬ 
held  some  technical  information? 

Yes _  No _ 

If  yes,  please  give  examples. 


11.  Do  you  feel  that  your  source  selection  was  affected  by 
a  contractor's  reluctance  to  reveal  technological 
information  in  his  proposal? 

Yes _  No _ 


If  yes,  please  explain  how  your  source  selection  was 
affected,  especially  in  terms  of  time  and  cost. 


12.  If  you  knew  that  a  contractor  withheld  technical 
information  from  his  technical  proposal  would  this 
lessen  his  chances  of  winning  the  contract? 

Yes _  No _ 

If  yes,  please  explain. 


13*  Do  you  feel  that  you  can  offer  contractors  the  level  of 
incentives  and  protection  necessary  to  insure  that 
state-of-the-art  technology  is  included  in  their  tech¬ 
nical  proposals? 

Yes _  No _ 

If  no  please  explain  what  incentives/protection  are 
necessary  to  insure  this. 


l4.  Are  there  any  comments  you  would  like  to  make  concerning 
contractor  technical  proposals  and  the  withholding  of 
some  technical  information/capability  from  those  pro¬ 
posals? 


15 •  Do  you  have  any  recommendations  in  regard  to  specific 
contractual  instruments  which  could  be  employed  or  de¬ 
veloped  to  insure  state-of-the-art  technology  is  not 
withheld  from  technical  proposals? 
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1.  How  many  proposals  (solicited  and  unsolicited)  did  you 
submit  to  the  Air  Force  during  FY  1979? 


2.  Of  the  proposals  which  were  submitted,  did  any  contain 
what  you  perceived  to  be  state-of-the-art  technology 
which  your  competitors  probably  did  not  possess? 

Yes _  No_ _ 


3.  a.  Has  the  possibility  that  anyone  may  obtain  copies  of 
your  technical  proposals  from  the  Air  Force  led  you 
to  refrain  from  submitting  a  proposal  on  a  procure¬ 
ment  which  you  otherwise  would  have  bid  upon? 

Yes _  No _ 


b.  Has  the  possibility  of  release  of  your  technical 
proposals  led  you  to  alter  your  final  price  in  a 
procurement? 

Yes _  No _ 


c.  Do  you  perceive  that  the  possibility  of  release  of 
technical  proposals  has  led  your  competition  to  alter 
their  final  price  in  a  procurement? 

Yes _  No _ 


d.  Has  the  possibility  of  release  of  your  technical 

proposals  substantially  affected  the  negotiation  time 
on  procurements  in  which  you  submitted  proposals? 

Yes _  No _ 
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4.  Do  you  identify  source  selection  sensitive  material  in 
your  technical  proposals  through  restrictive  legends  or 
other  means? 


Yes _  No _ 

If  yes,  what  type  of  material  do  you  consider  source 
selection  sensitive  material? 


5«  Do  you  know  of  instances  where  the  Government  has  re¬ 
leased  technical  proposals,  or  portions  thereof,  that 
you  have  submitted  that  contain  what  you  consider 
proprietary  technical  information? 

Yes _  No _ 


6.  Has  the  information  that  the  Air  Force  released  (Refer 
to  question  5)  enabled  another  company  to  compete  with 
you  in  procurements  in  which  they  could  not  compete 
prior  to  the  release? 

Yes _  No _ 

7.  Is  there  a  level  of  protection/incentives  which  the 
Government  must  offer  before  you  will  submit  a  proposal 
which  contains  source  selection  sensitive  data? 

Yes  No 

V”— 

If  yes,  please  define  that  level  of  protecticn/incentives 
which  you  require  or  deem  adequate  and  give  examples. 


8.  Do  you  have  any  recommendations  in  regard  to  contrac¬ 
tual  instruments  which  the  Government  should  employ  or 
develop  which  would  reduce  any  reluctance  you  may  have 
to  include  state-of-the-art  technology  in  your  technical 
proposals? 
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where » 


E(0)  = 

s  2  __  (nj-DS^  +  (n2-l)  S22 
c  (nj-1)  +  <n2-l) 

S2(D)  =  S  2  (-  +  -  ) 

'  '  c  'n2  n^' 

D  ~  ^2~^1  =  t(nt  +  n2-2) 

S(D) 

|i^  =  the  mean  of  the  group  of  contracts  that  had 
data  problems 

H2  a  the  mean  of  the  group  of  contracts  that  had  no 
data  problems 

n^  =  sample  of  data  problem  contracts 

n2  =  sample  of  no  data  problem  contracts 
2 

Sc  =  pooled  variance  estimator 

D  =  point  estimator  of  difference  of  two  sample  means 

p  - 

S  (D)  =  variance  estimator  of  D 
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APPENDIX  P 


COMPUTER  PROGRAM  AND  OUTPUT 


I 


0010##S,R(SL)  i,8,l6||,l6 

0020<ki  IDENTj  WP1186,  afit/ls  hubert/allen 

0030SJ  SELECT! SPSS/SPSS 

0040$ t  PRMFL !  DT ,  W ,  S , 80 A06 5/SPOUT 

0050RUN  NAMEi T-TEST  COMPARISON  OP  MEANS 

0060VARIABLE  LIST; COMP, COST, GROUP 

0070INPUT  FORMAT; PREEFIELD 

0080N  OF  CASES; 20 

0090VAR  LABELS; COMP, NUMBER  OF  COMPETING  OFFERS/ 

0100; COST, COST  OF  NEGOTIATION  PROCESS  IN  MILLIONS/ 

0110; GROUP, CONTRACT  GROUP 

01 20 VALUE  LABELS; GROUP  (1)  CONTRACTS  WITH  DATA  PROBLEMS/ 
0130; (2)  CONTRACTS  WITHOUT  DATA  PROBLEMS 
0140T-TEST;  GROUPS=GROUP  (2)/VARIABLES=C0MP 
0150READ  INPUT  DATA 
0160  1  3.1  1 
0170  2  17  1 
0180  2  17  1 
0190  3  0.06  1 
0200  3  0.1  1 
0210  3  0.28  1 
0220  2  -1.5  1 
0230  4  0.5  1 
0240  2  6.8  1 
0250  2  1 2.2  1 
0260  8  0.05  2 
0270  1  -98  2 
0280  1  -10  2 
0290  1  0.8  2 
0300  1  —33  2 
0310  7-62 
0320  1  -16  2 
0330  4  -0.5  2 
0340  4  -0.7  2 
0350  2  0.5  2 

0360T-TEST; GROUPS=GROUP  ( 2 )/VARI ABLES=COST 

0370FINISH 

0380$END  JOB 


SPSS  T-test  Program 
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. T-TEST-- 

GROUP  t  -  OROUP  GE  2. 

GROUP  2  -  GROUP  LT  2. 


VARIABLE 

NUMBER 

STANDARD 

STANDARD 

* 

* 

F 

2-TAIL 

OF  CASES 

MEAN 

DEVIATION 

ERROR 

* 

VALUE 

PROD. 

COMP  NUMBER  OF  COMPETING  OFFERS 

* 

GROUP  1 

10 

3.0000 

2.667 

0.843 

• 

* 

10.00 

0.002 

GROUP  2 

to 

2.4000 

0.843 

0.267 

* 

• 

POOLED  VARIANCE  ESTIMATE  *  SEPARATE  VARIANCE  ESTIMATE 

* 

T  DEGREES  OF  2-TAIL  *  T  DEGREES  OF  2-TAIL 
VALUE  FREEDOM  PROD.  *  VALUE  FREEDOM  PROB. 


* 

0.68  18  0.506  *  0.68  10.78  0.512 

* 

* 


Test  Results  For  Competition 
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VARIABLE 

NUMBER 

STANDARD 

STANDARD 

t 

$ 

F 

2-TAIL 

OF  CASES 

MEAN 

DEVIATION 

ERROR 

* 

VALUE 

PROB. 

COST  COST 

OF  NEGOTIATION  PROCESS  IN  MILLIONS 

* 

6R0UP  1 

10 

-13.0180 

30.374 

0.405 

* 

» 

17.41 

0.000 

GROUP  2 

to 

5.5340 

7.270 

2.302 

• 

• 

POOLED  VARIANCE  ESTIMATE  *  SEPARATE  VARIANCE  ESTIMATE 

* 

T  DEGREES  OF  2-TAIL  *  T  DEGREES  OF  2-TAIL 
VALUE  FREEDOM  PROD.  *  VALUE  FREEDOM  PROD. 


* 

* 

-1.88  18  0.076  •  -1.88  10.03  0.080 


Test  Results  For  Cost 
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